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6/6/1 (Item 1 from file: 2) 

6512318 INSPEC Abstract Number: C2000-04-1340G-003 
Title: Controlled Markov processes and mathematical finance 

Publication Date: 1 999 

Copyright 2000, lEE 
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...examined 50 records 
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telecommunications accounts 
Publication Year: 1 996 



9/6/2 (Item 1 from file: 34) 

05907269 Genuine Article#: XF842 Number of References: 1 52 
Title: Maximization and the act of choice (ABSTRACT AVAILABLE) 
Publication date: 1 9970700 



9/6/3 (Item 2 from file: 34) 

04782490 Genuine Article*: UG531 Number of References: 43 
Title: AN INFORMATION-THEORETIC TECHNIQUE TO DESIGN BELIEF NETWORK-BASED 
EXPERT-SYSTEMS (Abstract Available) 



9/6/4 (Item 3 from file: 34) 

02998107 Genuine Article*: MX917 Number of References: 46 

Title: UNCERTAINTY ANALYSIS APPLIED TO SUPERVISED CONTROL OF APHIDS AND 

BROWN RUST IN WINTER-WHEAT .1. QUANTIFICATION OF UNCERTAINTY IN 

COST-BENEFIT CALCULATIONS (Abstract Available) 



9/6/5 (Item 4 from file: 34) 

02281 1 68 Genuine Article*: KQ1 76 Number of References: 24 

Title: ON KNOWLEDGE REPRESENTATION IN BELIEF NETWORKS (Abstract Available) 



9/6/6 (Item 1 from file: 647) 

01 1 36279 CMP ACCESSION NUMBER: IWK1 9970825S0036 

the Future is Now - Leading-edge IT managers tap into vendor R&D efforts 

PUBLICATION DATE: 970825 

WORD COUNT: 3208 

?t s9/7/1,3,5 

9/7/1 (Item 1 from file: 8) 
DlALOG(R)File 8:Ei Compendex(R) 
(c) 2000 Engineering Info. Inc. All rts. reserv. 

04536692 E.I. No: EIP961 00378405 

Title: Constructing Bayesian networks to predict uncollectible 
telecommunications accounts 

Author: Ezawa, Kazuo J.; Norton, Steven W, 

Corporate Source: AT&T Consumer Lab, Murray Hill, NJ, USA 

Source: IEEE Expert v 1 1 n 5 Oct 1996. p 45-51 

Publication Year: 1996 

CODEN: IEEXE7 ISSN: 0885-9000 

Language: English 

Document Type: JA; (Journal Article) Treatment: A; (Applications); T; 
(Theoretical) 
Journal Announcement: 961 2W4 

Abstract: The complexities of building models that can predict whether a 
customer account or transaction is collectible are greater than most 
current learning systems can handle. This article describes software that 
builds Bayesian network models for such prediction. The Advanced Pattern 
Recognition and Identification (APRl) system's key strength is its ability 
to efficiently select relevant variables and dependencies to build 
conditionally dependent models. In fact, APRl reads the data from 
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secondary storage at most five times during the entire model-building 
process. This is in sharp contrast to other Bayesian network learning 
systems, whose complexity grows linearly or quadratically with the number 
of input variables, 10 Refs. 



9/7/3 (Item 2 from file: 34) 
DIALOG(R)File 34:SciSearch(R) Cited Ref Sci 
(c) 2000 Inst for Sci Info. All rts. reserv. 

04782490 Genuine Article*: UG531 Number of References: 43 

Title: AN INFORMATION-THEORETIC TECHNIQUE TO DESIGN BELIEF NETWORK-BASED 

EXPERT-SYSTEMS 
Author(s): SARKAR S; SRIRAM RS; JOYKUTTY S; MURTHY I 

Corporate Source: LOUISIANA STATE UNIV,COLL BUSINESS ADM,DEPT INFORMAT SYST 
& DECIS SCI/BATON ROUGE//LA/70803; GEORGIA STATE UNIV,SCH 
ACCOUNTANCY/ATLANTA//GA/30303 

Journal: DECISION SUPPORT SYSTEMS, 1996, VI 7, N1 (APR 22), PI 3-30 

ISSN: 0167-9236 

Language: ENGLISH Document Type: ARTICLE 

Abstract: This paper addresses the problem of constructing belief network 
based expert systems. We discuss a design tool that assists in the 
development of such expert systems by comparing alternative 
representations. The design tool uses information theoretic measures to 
compare alternative structures. Three important capabilities of the 
design tool are discussed: (i) evaluating alternative structures based 
on sample data; (ii) finding optimal networks with specified 
connectivity conditions; and (iii) eliminating weak dependencies from 
derived network structures. We have examined the performance of the 
design tool on many sets of simulated data, and show that the design 
tool can accurately recover the important dependencies across variables 
in a problem domain. We illustrate how this program can be used to 
design a belief network for evaluating the financial distress situation 
for banks. 



9/7/5 (Item 4 from file: 34) 
DIALOG(R)File 34:SciSearch(R) Cited Ref Sci 
(c) 2000 Inst for Sci Info, All rts. reserv. 

02281 1 68 Genuine Article*: KQ1 76 Number of References: 24 
Title: ON KNOWLEDGE REPRESENTATION IN BELIEF NETWORKS 
Author(s): ABRAMSON B 

Corporate Source: UNIV SO CALIF,DEPT COMP SCI/LOS ANGELES//CA/90089 
Journal: LECTURE NOTES IN COMPUTER SCIENCE, 1991, V521, P86-96 
ISSN: 0302-9743 

Language: ENGLISH Document Type: ARTICLE 

Abstract: Three focal elements of knowledge-based system design are (i) 
acquiring information from an expert, (ii) representing the information 
in a system-usable form, and (iii) using the information to draw 
inferences about specific problem instances. In the artificial 
intelligence (Al) literature, the first element is referred to as 
knowledge acquisition, while the second and third are embodied in a 
system's knowledge base and inference engine, respectively. Al, 
however, is not alone in its concern for these issues. Researchers in 
several of the statistical decision sciences, notably decision analysis 
(DA), have also investigated them. This paper discusses the use of 
belief networks-a formalism that lies somewhere between Al and DA-as an 
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overall framework for knowledge-based systems. Unlike previous work, 
which has concentrated on either the networks' mathematical properties 
or on their implementation as a specific system, this paper is oriented 
towards the concerns of general system design. Concrete examples are 
drawn from one medical system (Pathfinder) and from one financial 
system (ARCOl), and in particular, from a consideration of their 

similarities and differences. The design principles abstracted from 
these systems suggests a powerful, coherent design philosophy guided by 
the simple thought: form follows function. 

?s s8 and financ? 
5088 S8 
259164 FINANC? 

510 46 S8 AND FINANC? 
?s slO and (risk or derivative?) 

46 S10 
727460 RISK 
848081 DERIVATIVE? 

51 1 6 SI 0 AND (RISK OR DERIVATIVE?) 
?s7 and caus? 

Processed 10 of 16 files... 
Completed processing all files 
2483394 7 
1793209 CAUS? 
SI 2 157291 7 AND CAUS? 
?s 7 and caus? 
2483394 7 
1793209 CAUS? 
SI 3 157291 7 AND CAUS? 
?s s7 and caus? 

1563 S7 
1793209 CAUS? 
SI 4 97 S7 AND CAUS? 
?s si 1 not s9 

6 S11 
6 S9 

SI 5 0 S11 N0TS9 
?t si 4/6/1 -97 

14/6/1 (Item 1 from file: 2) 

6445719 INSPEC Abstract Number: C2000-02-1 290D-038 
Title: A new stochastically flexible event methodology with application to 
Proposition 103 

Publication Date: 1 6 Nov. 1 999 

Copyright 1999, lEE 



14/6/2 (Item 2 from file: 2) 

6247982 INSPEC Abstract Number: CI 999-06-1 290D-090 

Title: From ruin theory to pricing reset guarantees and perpetual put 
options 

Publication Date: 31 March 1999 
Copyright 1 999, lEE 



14/6/3 (Item 3 from file: 2) 

5477709 INSPEC Abstract Number: B9702-8260-01 6 
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Title: Fatigue reliability of wind turbine fleets: the effect of 
uncertainty on projected costs 
Publication Date: Nov. 1 996 
Copyright 1997, EE 



14/6/4 (Item 4 from file: 2) 

5206589 INSPEC Abstract Number: C9604-71 60-051 
Title: An innovative decision support system for reliability assessment of 
management structures of organizations in the construction industry 

Publication Date: 1 995 

Copyright 1 996, lEE 



14/6/5 (Item 5 from file: 2) 

5193072 INSPEC Abstract Number: C9604-71 65-002 

Title: Forecasting short term regional gas demand using an expert system 

Publication Date: 1 996 

Copyright 1 996, lEE 



14/6/6 (Item 6 from file: 2) 

4995552 INSPEC Abstract Number: B9508-01 60-009 
Title: From determinism to probabilism 

Publication Date: Sept.-Oct. 1 995 

Copyright 1 995, lEE 



14/6/7 (Item 7 from file: 2) 

4973864 INSPEC Abstract Number: B9507-01 70N-041 
Title: Using R&M analysis to calculate economic risk in the process 

Industries 
Publication Date: 1 995 
Copyright 1995, lEE 



14/6/8 (Item 8 from file: 2) 

4931938 INSPEC Abstract Number: C9506-1 290D-006 
Title: Martingale analysis for assets with discontinuous returns 

Publication Date: Feb. 1 995 

Copyright 1 995, lEE 



14/6/9 (Item 9 from file: 2) 

4585647 INSPEC Abstract Number: C9403-61 70-026 
Title: Examples of causal probabilistic expert systems 
Publication Date: 1 993 



14/6/10 (Item 10 from file: 2) 

03361217 INSPEC Abstract Number: A89053325, B89035482 
Title: Functional perspective on risk analysis in nuclear power 
Publication Date: 1 987 



14/6/11 (Item 11 from file: 2) 
03029378 INSPEC Abstract Number: B88000268 
Title: A RAM evaluation tool: combining abridged-PRA with root-cause 
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analysis 
Publication Date: 1 987 



14/6/12 (Item 1 from file: 6) 
1 950853 NTIS Accession Number: DE96004308 

Fatigue reliability of wind turbine fleets: The effect of uncertainty of 
projected costs 

1995 



14/6/13 (Item 2 from file: 6) 
1768953 NTIS Accession Number: AD-A270 711/5 

Economic Analysis of Costs Incurred from Chemical Exposures in the 
Workplace Resulting in Non-Carcinogenic Responses as Additional 
Justification for Pollution Prevention Projects 

(Master's thesis) 

26 Aug 93 



14/6/14 (Item 3 from file: 6) 
1 747463 NTIS Accession Number: DE93621 309 

Foersaekringspremier foer svenska kaernkraftverk, (Insurance cost of 
Swedish nuclear power plants) 

1992 



14/6/15 (Item 4 from file: 6) 
1 61 6733 NTIS Accession Number: DE920001 32 
When is a dose not a dose 
1991 



14/6/16 (Item 5 from file: 6) 

1 021 563 NTIS Accession Number: NUREG/CR-1 1 20-V1 0 
Seismic Safety Margins Research Program 
(Progress rept. no. 14) 
Jan 83 



1 4/6/1 7 (Item 6 from file: 6) 

0644752 NTIS Accession Number: PB-268 926/3/XAB 

Financing Minority Enterprise via the Economic Opportunity Loan Program: 
An Evaluation 

(Discussion papers) 

Aug 74 



14/6/18 (Item 1 from file: 8) 
03472890 

Title: Problems in the economic evaluation of proposed projects in 
emerging energy technologies. 
Conference Title: AlChE National Meeting 
Publication Year: 1 991 



14/6/19 (Item 2 from file: 8) 
02098532 
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Title: EXPERT SYSTEM FOR FOREIGN CURRENCY HEDGING. 

Conference Title: 1 985 ACM Thirteenth Annual Computer Science Conference. 

Publication Year: 1 985 



14/6/20 (Item 3 from file: 8) 
01354788 

Title: Risk Analysis Oriented towards Financial and Technical Control. 
Title: L'ANALYSE DE RISQUE VERS UNE MAITRISE FINANCIERE ET TECHNIQUE. 
Publication Year: 1 982 



14/6/21 (Item 4 from file: 8) 
01165036 

Title: MANAGING RISKS PART OF SUCCESS. 
Publication Year: 1 982 



14/6/22 (Item 1 from file: 34) 

08413949 Genuine Article*: 282ZB Number of References: 35 

Title: Nitrogen and phosphorous management on Wisconsin farms: Lessons 

learned for agricultural water quality programs (ABSTRACT AVAILABLE) 
Publication date: 20000000 



14/6/23 (Item 2 from file: 34) 

08250374 Genuine Article*: 262XL Number of References: 41 
Title: Which statin Is most efficient for the treatment of 

hypercholesterolemia? A cost effectiveness analysis (ABSTRACT 
AVAILABLE) 
Publication date: 19991100 



14/6/24 (Item 3 from file: 34) 

07559345 Genuine Article*: 1 81 FE Number of References: 1 1 6 
Title: Risk and dangerousness (ABSTRACT AVAILABLE) 
Publication date: 19990300 



14/6/25 (Item 4 from file: 34) 

070981 64 Genuine Article*: 1 24CW Number of References: 59 
Title: The Seinhorst Research Program (ABSTRACT AVAILABLE) 
Publication date: 1 9980900 



14/6/26 (Item 5 from file: 34) 

07005737 Genuine Article*: 11400 Number of References: 16 
Title: Exceptional circumstances provisions in Australia - is there too 

much emphasis on drought? (ABSTRACT AVAILABLE) 
Publication date: 19980700 



14/6/27 (Item 6 from file: 34) 

061 5421 5 Genuine Article*: XY695 Number of References: 75 

Title: Farm decision making under risk and uncertainty (ABSTRACT AVAILABLE 

) 

Publication date: 1 9970700 
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14/6/28 (Item 7 from file: 34) 

05866702 Genuine Article*: XD047 Number of References: 53 
Title: Stressful life events and genetic liability to major depression: 

Genetic control of exposure to the environment (ABSTRACT AVAILABLE) 
Publication date: 19970500 



14/6/29 (Item 8 from file: 34) 

0561 7042 Genuine Article#: WK91 2 Number of References: 1 2 

Title: A prophylactic implantable cardioverter-defibrillator? (ABSTRACT 

AVAILABLE) 
Publication date: 19960912 



14/6/30 (Item 9 from file: 34) 

05371 1 23 Genuine Article*: VU201 Number of References: 44 
Title: RISK PROPENSITY AND FIRM PERFORMANCE - A STUDY OF THE 
PETROLEUM-EXPLORATION INDUSTRY (Abstract Available) 



14/6/31 (Item 10 from file: 34) 

0511 2747 Genuine Article*: VB050 Number of References: 20 
Title: PREDICTING THE PRESENT VALUE DISTRIBUTION OF A FOREST PLANTATION 
INVESTMENT (Abstract Available) 



14/6/32 (Item 11 from file: 34) 

05074634 Genuine Article*: TN531 Number of References: 37 
Title: THE PRODUCTIVITY OF FINANCIAL INTERMEDIATION AND THE TECHNOLOGY OF 
FINANCIAL PRODUCT MANAGEMENT (Abstract Available) 



1 4/6/33 (Item 1 2 from file: 34) 

04145498 Genuine Article*: RH71 5 Number of References: 37 
Title: THE STRUCTURAL RELATIONSHIP BETWEEN FINANCIAL RATIOS AND 
CAPITAL-ASSET PRICING (Abstract Available) 



14/6/34 (Item 13 from file: 34) 

03865509 Genuine Article*: QM820 Number of References: 12 
Title: APPARENTLY STABLE INCREMENTS IN FINANCE DATA - COULD ARCH EFFECTS BE 
THE CAUSE (Abstract Available) 



14/6/35 (Item 14 from file: 34) 

03506818 Genuine Article*: PJ325 Number of References: 33 
Title: BENEFIT TRANSFER PROTOCOL FOR LONG-TERM HEALTH RISK VALUATION - A 
CASE OF SURFACE-WATER CONTAMINATION (Abstract Available) 



14/6/36 (Item 15 from file: 34) 

02281 1 68 Genuine Article*: KQ1 76 Number of References: 24 

Title: ON KNOWLEDGE REPRESENTATION IN BELIEF NETWORKS (Abstract Available) 



14/6/37 (Item 1 from file: 94) 

04147781 JICST ACCESSION NUMBER: 99A0538729 FILE SEGMENT: JICST-E 
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Acceptance of Long-term Probabilistic Forecast of Earthquake and its Social 
Meanings., 1 999 



1 4/6/38 (Item 2 from file: 94) 

04113525 JICST ACCESSION NUMBER: 99A0441625 FILE SEGMENT: JICST-E 
Efficiency in the Japanese Life Insurance Market and Advertising., 1 998 



1 4/6/39 (Item 3 from file: 94) 

02347686 JICST ACCESSION NUMBER: 95A0549029 FILE SEGMENT: JICST-E 
The Development of Forestry Manegment and the Strategies for Financial and 
Resources Crisis: the Case of IGA Forestry Enterprise,, 1995 



14/6/40 (Item 4 from file: 94) 

00883101 JICST ACCESSION NUMBER: 89A0243178 FILE SEGMENT: JICST-E 
Development of corporate risk analyzing system based on financial data., 
1988 



14/6/41 (Item 1 from file: 103) 
04131262 SPN-97-0B0068; EDB-97-039966 
Title: Current trends towards a new regulation and evolution of fire 

protection systems technologies in nuclear power plants 
Original Title: Tendencias actuales hacia una nueva regulacion y evolucion 

de la tecnica de los sistemas de proteccion contra incendios en 

centrales nucleares 
Title: Papers presented at Congresses and Conferences: 1 995 
Original Title: Ponencias presentadas a Congresos y conferencias: 1 995 
Publication Date: 1996 



1 4/6/42 (Item 2 from file: 1 03) 
03952329 SPN-96-0B01 26; EDB-96-036089 
Title: Probabilistic safety analysis applied to RBMK reactors 
Original Title: Estudio probabilista de la seguridad de los reactores RBMK 
Title: Papers presented at congresses and Conferences: 1 994 
Original Title: Ponencias presentadas a Congresos y Conferencias: 1 994 
Publication Date: 1995 



14/6/43 (Item 3 from file: 103) 
03819382 EDB-95-063150 

Title: Barycentric approximation in financial decision making 

Title: Mathematical programming: State of the art 1 994 

Conference title: 1 5. international symposium on mathematical programming 

Publication Date: 1994 



14/6/44 (Item 4 from file: 103) 

03769984 NEDO-94-91 2975; INS-95-001 245; EDB-95-01 3752 

Title: Organization and management activities in the nuclear power industry 

Title: Proceedings of the Twenty-First Water Reactor Safety Information 

Meeting: Volume 1, Plenary session; Advanced reactor research; 

advanced control system technology; advanced instrumentation and 

control hardware; human factors research; probabilistic risk 

assessment topics; thermal hydraulics; thermal hydraulic research for 
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advanced passive LWRs 
Conference title: 21. water reactor safety information meeting 
Publication Date: Apr 1994 



14/6/45 (Item 5 from file: 103) 
03759391 EDB-95-003159 

Title: Experimental evaluation of traffic load on burled gas pipelines 
Title: First world conference on structural control (1WCSC) 
Conference title: 1. world conference on structural control (WCSC-1) 
Publication Date: 1994 



14/6/46 (Item 6 from file: 103) 
03474448 AIX-24-033960; EDB-93-053324 
Title: Insurance cost of Swedish nuclear power plants 
Original Title: Foersaekringspremier foer svenska kaernkraftverk 
Publication Date: 1992 



14/6/47 (Item 7 from file: 103) 
03135294 EDB-91 -072729 

Title: Total Probable Risk analysis: A technique for quantitative risk 

evaluation of hazardous waste disposal options 
Publication Date: Sum 1 989 



14/6/48 (Item 8 from file: 103) 
02790833 AIX-21-01 0295; EDB-90-008046 
Title: The characteristics of risks of major disasters 
Publication Date: 8 Aug 1 989 



14/6/49 (Item 9 from file: 103) 
02789546 AIX-21 -003606; EDB-90-006759 
Title: Radiological risks and civil liability 
Publication Date: 1989 



14/6/50 (Item 10 from file: 103) 
02066285 INS-88-000255; EDB-88-009005 
Title: Price- Anderson Law - reports on Price-Anderson issues 
Publication Date: 1985 



14/6/51 (Item 11 from file: 103) 
01 31 7423 EPA-1 0-000953; EDB-84-01 5096 

Title: Effect of tax, financing, and operating-cost incentives on retiree 
homeowners' current and potential decisions to purchase energy-saving 
improvements 

Publication Date: 1983 



14/6/52 (Item 12 from file: 103) 
01302593 EDB-84-000264 
Title: Portfolio analysis of exploration strategies 
Conference title: SPE hydrocarbon economics and evaluation symposium 
Publication Date: Mar 1983 
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14/6/53 (Item 13 from file: 103) 
01199273 ERA-08-032962; EDB-83-099310 
Title: Industrial viewpoint of synthetic fuels 

Title: Proceedings of the eighth underground coal conversion symposium 
Conference title: 8. underground coal conversion symposium 
Publication Date: Nov 1 982 



14/6/54 (Item 1 from file: 144) 
1 4244226 PASCAL No.: 99-0446845 

Decision tree analysis and risk modeling to appraise investments on major 
oil field projects 

MEGS 99 : managing the future : challenges for people, resources and 
technology : Bahrain, 20-23 February 1 999 
1999 

Copyright (c) 1999 INIST-CNRS. All rights reserved. 



14/6/55 (Item 2 from file: 144) 
1 2761 303 PASCAL No.: 96-0475307 

A prophylactic implantable cardloverther-defibrillator ? Discussion 
1996 

Copyright (c) 1996 INIST-CNRS. All rights reserved. 



14/6/56 (Item 3 from file: 144) 
1 2747827 PASCAL No.: 96-0461 032 

Eliciting von Neumann-Morgenstern utilities when probabilities are 
distorted or unknown 
1996 

Copyright (c) 1996 INIST-CNRS. All rights reserved. 



14/6/57 (Item 4 from file: 144) 
1 21 32073 PASCAL No.: 95-0364062 

Der verhaltenssteuernde Einfluss der Arzthaftung: Auswirkungen auf die 
Technologiewahl 

(The Influence of the physician's professional indemnity on his attitude 
and behaviour: effects on the choice of technology) 
1995 



14/6/58 (Item 5 from file: 144) 
01 390748 PASCAL No.: 77-044531 0 

PROTECTION AGAINST LOW PROBABILITY EVENTS: A SEQUENTIAL MODEL OF CHOICE. 
IN: MODELING AND SIMULATION. ANNU. PITTSBURGH CONF. 7. PROC; PITTSBURGH; 
1976 
1976 



1 4/6/59 (Item 1 from file: 239) 
16622524 MR 2000a#91060 

On the discounted penalty at ruin in a jump-diffusion and the perpetual 
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put option. 
1998 



1 4/6/60 (Item 2 from file: 239) 
1 6427964 MR 98e#60004 

Athens Conference on Applied Probability and Time Series Analysis. Vol. 

I. 

Applied probability. In honor of J. M. Gani. Papers from the conference 
held in Athens, March 22-26, 1995. Edited by C. C. Heyde, Yu. V. Prohorov 
[Yu. V. Prokhorov], R. Pyke and S. T. Rachev, 
1996 



14/6/61 (Item 3 from file: 239) 
16314494 MR 97e#62133 

Delay in claim settlement and ruin probability approximations. 

14/6/62 (Item 4 from file: 239) 
16139891 MR 95j#90025 
Complex economic dynamics. Vol. I. 

An introduction to dynamical systems and market mechanisms. With a 
foreword by Paul A. Samuelson. 
1994 



14/6/63 (Item 5 from file: 239) 
03133582 MR 89j#90002 
Okonomie und Mathematik, 
Economics and mathematics 

Rudolf Henn zum 65. Geburtstag. [To Rudolf Henn on his 65th birthday] 
Edited by 0. Opitz and B. Rauhut. 
1987 



14/6/64 (Item 6 from file: 239) 
03032624 MR88b#62193 

Cost-optimal performance of special reliability tests using prior 
knowledge. 
1986 



14/6/65 (Item 7 from file: 239) 
02522175 MR 81g#62147 

Nonparametric estimation of functions in a model of competing risks from 
incomplete longitudinal data. 
1979 



14/6/66 (Item 1 from file: 241) 
1 065093 SUBFILE: EPRI TECHNICAL REPORT 

Property Damage Risk Assessment Scoping Study 

REPORT NUMBER: EPRI TR-1 08261 01 52p. 

CONTRACT/GRANT NO.: WO3719-04 

PUBLICATION YEAR: 1 997 07 
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14/6/67 (Item 2 from file: 241) 
1 064902 SUBFILE: EPRI TECHNICAL REPORT 

Nuclear Property Insurance Study 

REPORT NUMBER: EPRI TR-1 08061 0026p. 

CONTRACT/GRANT NO.: W03719-04; WO3719-08; RP3719-04; RP3719-08 
PUBLICATION YEAR: 1 997 06 



14/6/68 (Item 3 from file: 241) 

1 0421 34 EPRI ACCESSION NO: 1 633200 SUBFILE: EPRI TECHNICAL REPORT 

Capital Requirements for the U.S. Investor-Owned Electric Utility 
Industry: 1 985-2005 

REPORT NUMBER: EPRI P-5830 OlOOp. 

CONTRACT/GRANT NO.: RP1 920-03 

PUBLICATION YEAR: 1 988 06 



14/6/69 (Item 1 from file: 647) 

01 20461 5 CMP ACCESSION NUMBER: EBN1 9991 1 1 5S0020 
Chip stocks could correct next year (Wall Street Watch) 
PUBLICATION DATE: 991115 
WORD COUNT: 361 



1 4/6/70 (Item 2 from file: 647) 

01 1 99786 CMP ACCESSION NUMBER: NWC1 9990906S001 6 
Orchestrating Today's E-Commerce 
PUBLICATION DATE: 990906 
WORD COUNT: 5342 



14/6/71 (Item 3 from file: 647) 

01 1 95905 CMP ACCESSION NUMBER: EBN1 999071 2S0003 

Exploring Alternatives - A Growing Number Of Sites Offer A Variety Of 

Services, Ranging From Online Auctions To Detailed Catalogs To 

Comparative Technical Data. 
PUBLICATION DATE: 990712 
WORD COUNT: 10559 



1 4/6/72 (Item 4 from file: 647) 

01 1 9091 2 CMP ACCESSION NUMBER: LTHl 9990503S0040 

The Wolf At The Door - Service Providers Keep Building More Secure VPNs, 

And Keep Getting Blown Away By Cost, Complexity And The Criminal Mind. 
PUBLICATION DATE: 990503 
WORD COUNT: 2334 



14/6/73 (Item 5 from file: 647) 

01 1 90735 CMP ACCESSION NUMBER: IWK1 9990503S0041 
Get A Grasp On Knowledge 
PUBLICATION DATE: 990503 
WORD COUNT: 2548 



14/6/74 (Item 6 from file: 647) 

01 1 83493 CMP ACCESSION NUMBER: NWC1 99901 25S0009 
The Power of Knowledge and Information (Net Results) 
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PUBLICATION DATE: 990125 
WORD COUNT: 1 520 



14/6/75 (Item 7 from file: 647) 

01 1 80252 CMP ACCESSION NUMBER: CRN1 9981 207S01 90 

The One That Got Away - HP's Innovation Has Slipped Off The Hook. Without 

It, Company Chairman And CEO Lewis Piatt Could Find It Difficult To 

Lure Reseller, Corporate, And Wall Street Interest 
PUBLICATION DATE: 981207 
WORD COUNT: 1 574 



14/6/76 (Item 8 from file: 647) 

01 1 74864 CMP ACCESSION NUMBER: EBN1 9981 01 2S0045 
IC design gap feeding semiconductor glut (Executive Comment) 
PUBLICATION DATE: 981012 
WORD COUNT: 647 



1 4/6/77 (Item 9 from file: 647) 

01 1 72488 CMP ACCESSION NUMBER: INWl 998091 4S0002 
Cautiously, CEOs Lead The Way 
PUBLICATION DATE: 980914 
WORD COUNT: 2249 



1 4/6/78 (Item 1 0 from file: 647) 

01 1 57795 CMP ACCESSION NUMBER: INWl 9980406S0004 
Linking The Supply Chain With The Cash Register (Web Value Chain) 
PUBLICATION DATE: 980406 
WORD COUNT: 2364 



1 4/6/79 (Item 1 1 from file: 647) 

01 1 36279 CMP ACCESSION NUMBER: IWK1 9970825S0036 

the Future is Now - Leading-edge IT managers tap into vendor R&D efforts 

PUBLICATION DATE: 970825 

WORD COUNT: 3208 



1 4/6/80 (Item 1 2 from file: 647) 

01 1 30802 CMP ACCESSION NUMBER: IWK1 9970707S0046 

Visible Means Of Support - Building a support architecture for enterprise 

systems before they roll out reduces the risk of project failure 
PUBLICATION DATE: 970707 
WORD COUNT: 2162 



14/6/81 (Iterti 13 from file: 647) 

01 1 24892 CMP ACCESSION NUMBER: CWK1 9970505S0040 
YEAR 2000 COUNTDOWN - Protect Your Net! 
PUBLICATION DATE: 970505 
WORD COUNT: 2068 



1 4/6/82 (Item 1 4 from file: 647) 

01 1 03771 CMP ACCESSION NUMBER: VAR1 996091 5S0031 
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Building The Enterprise Network - Internetworking offers big-ticket 
product sales and high-service revenue for VARs who can handle its 
challenges 

PUBLICATION DATE: 960915 

WORD COUNT: 1 640 



14/6/83 (Item 15 from file: 647) 

01 1 0251 4 CMP ACCESSION NUMBER: CRN1 9960909S0096 
Internet security: What measures can be taken? 
PUBLICATION DATE: 960909 
WORD COUNT: 8909 



14/6/84 (Item 16 from file: 647) 

01 097555 CMP ACCESSION NUMBER: VAR1 996071 5S0037 
Lotus-Notable For Its Groupware Platform - At the heart of IBM's 

network-centric computing strategy, Notes is worth a gamble 
PUBLICATION DATE: 960715 
WORD COUNT: 721 



14/6/85 (Item 17 from file: 647) 

01 0801 60 CMP ACCESSION NUMBER: IWK1 99601 29S0033 

Disaster Prevention - New Software Fights Fires - Insurance cooperative 

helps telcos reduce risk of network damage 
PUBLICATION DATE: 960129 
WORD COUNT: 855 



1 4/6/86 (Item 1 8 from file: 647) 

01 065805 CMP ACCESSION NUMBER: EBN1 9950925S0088 
Top Purchasers - The Heavy Hitters (Opinion) 
PUBLICATION DATE: 950925 
WORD COUNT: 2976 



14/6/87 (Item 19 from file: 647) 

01 0591 00 CMP ACCESSION NUMBER: CRN1 995071 7S01 34 

Channel Exec Forum Mulls Win 95 Rollout Challenges - Some expect limited 

corporate migration within 90 days (a look inside) 
PUBLICATION DATE: 950717 
WORD COUNT: 1 796 



14/6/88 (Item 20 from file: 647) 

01 035784 CMP ACCESSION NUMBER: NWC1 9941 1 1 5S001 9 
The ATM Deskset 
PUBLICATION DATE: 941115 
WORD COUNT: 1016 



14/6/89 (Item 21 from file: 647) 

01022971 CMP ACCESSION NUMBER: CRN1 994051 6S271 4 

Portrait of Japanese computer companies - Juggling cultures in a global 

marketplace 
PUBLICATION DATE: 940516 
WORD COUNT: 5365 
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14/6/90 (Item 22 from file: 647) 

006451 1 8 CMP ACCESSION NUMBER: CRN1 9890424S1 990 
Businessland Upside Potential Limited Near Term 
PUBLICATION DATE: 890424 
WORD COUNT: 1 994 



14/6/91 (Item 23 from file: 647) 

0064241 7 CMP ACCESSION NUMBER: IWK1 989081 4S2964 

Unum: Divide And Conquer - A huge database enabled this insurer to build 

on niches In the market 
PUBLICATION DATE: 890814 
WORD COUNT: 1 574 



14/6/92 (Item 24 from file: 647) 

00641 330 CMP ACCESSION NUMBER: IWK1 9891 002S1 873 

EIS Powers Executives - America's corporate top brass is flocking to Ann 

Arbor to see if Comshare's system lives up to its reputation 
PUBLICATION DATE: 891002 
WORD COUNT: 3509 



14/6/93 (Item 25 from file: 647) 

00637568 CMP ACCESSION NUMBER: CWK1 9890424S1 508 
Peter Sevcik compares a carrier who sells hybrid network 

management-management of combined public and private networks-to a 

t... 

PUBLICATION DATE: 890424 
WORD COUNT: 1 543 



14/6/94 (Item 26 from file: 647) 

00607500 CMP ACCESSION NUMBER: CRN1 991 1 01 4S291 6 
JANUARY 1991 
PUBLICATION DATE: 911014 
WORD COUNT: 4744 



14/6/95 (Item 27 from file: 647) 

00566792 CMP ACCESSION NUMBER: CWK1 990031 9S2263 
Talking About Integration 
PUBLICATION DATE: 900319 
WORD COUNT: 5109 



14/6/96 (Item 28 from file: 647) 

00546791 CMP ACCESSION NUMBER: CRN1 993071 2S51 49 
MOVING AHEAD IN THE 1 990s - Execs debate tech support 
PUBLICATION DATE: 930712 
WORD COUNT: 5257 



14/6/97 (Item 29 from file: 647) 

00517762 CMP ACCESSION NUMBER: 0ST1 9920427S1 91 9 
A Sun Exec Explains Sun Pro 
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PUBLICATION DATE: 920427 
WORD COUNT: 4504 



?t 514/7/9,19,20,21,24,40 

1 4/7/9 (Item 9 from file: 2) 
DIAL0G(R)File 2:INSPEC 

(c) 2000 Institution of Electrical Engineers. All rts. reserv, 

4585647 INSPEC Abstract Number: C9403-61 70-026 
Title: Examples of causal probabilistic expert systems 
Author(s): Noormohammadian, M.; Oppel, U.G. 

Author Affiliation: Math. Inst. & Ludwig, Maximilians-Univ., Munchen, 
Germany 

Conference Title: Symbolic and Quantitative Approaches to Reasoning and 
Uncertainty. European Conference ECSQARU '93 Proceedings p. 290-5 
Editor(s): Clark, M.; Kruse, R.; Moral, S. 
Publisher: Springer-Verlag, Berlin, Germany 

Publication Date: 1 993 Country of Publication: West Germany x+390 pp. 
ISBN: 3 540 57395 X 

Conference Date: 8-1 0 Nov. 1 993 Conference Location: Granada, Spain 
Language: English Document Type: Conference Paper (PA) 
Treatment: Applications (A); Theoretical (T) 

Abstract: Presents examples of expert systems which are based on single 
causal probability network (CPNs) and on sequences of adapted and 
controlled CPNs and which are constructed and operated using the shell 
HUGIN. They are applied to risk analysis in genetics, diagnosis of the 
stock market, claims reserving, evaluation of multisensor systems, and 
diagnosis and prognosis of metabolic processes. (13 Refs) 



14/7/19 (Item 2 from file: 8) 
DlALOG(R)File 8:Ei Compendex(R) 
(c) 2000 Engineering Info. Inc. All rts. reserv. 

02098532 E.l. Monthly No: EIM8606-040394 
Title: EXPERT SYSTEM FOR FOREIGN CURRENCY HEDGING. 
Author: Star, Spencer 
Corporate Source: Univ Laval, Que, Can 

Conference Title: 1 985 ACM Thirteenth Annual Computer Science Conference. 
Conference Location: New Orleans, LA, USA Conference Date: 19850312 
Sponsor: ACM, New York, NY, USA 
E.l. Conference No.: 071 1 3 
Source: Pub! by ACM, New York, NY, USA p 421 
Publication Year: 1 985 
ISBN: 0-89791-150-4 
Language: English 

Document Type: PA; (Conference Paper) 
Journal Announcement: 8606 

Abstract: International corporations present their financial statements 
in terms of a single currency, often based on the currency of the home 
office or on the dollar, even though a large part of their foreign 
operations are completed using other currencies. Unancticipated moves in 
the volatile foreign exchange markets can severely distort the year-end 
results, sometimes causing large losses in an otherwise profitable year. 
Recent innovations in the markets for foreign currency futures and options 
allow corporations to implement complex hedging strategies to reduce 
foreign exchange risk. A good hedging strategy requires both a statistical 
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analysis and expert judgment as to how to combine this analysis with 
specific corporate foreign currency demands and the desire for reduced 
risk. This paper is a progress report on our attempt to build an expert 
system to make decisions on hedging strategies that reduce foreign exchange 
risk. An original aspect of our approach is that we will analyze the 
reactions of experts to a simulated market to determine the rules in our 
knowledge base. The expert system's performance will be verified by 
analyzing its decisions in a simulated market, (Author abstract) 



14/7/20 (Item 3 from file: 8) 
DIALOG(R)File 8:Ei Compendex(R) 
(c) 2000 Engineering Info. Inc. All rts. reserv. 

01 354788 E.I. Monthly No: EI8305032738 E.I. Yearly No: EI8301 91 53 
Title: Risk Analysis Oriented towards Financial and Technical Control. 
Title: ^ANALYSE DE RISQUE VERS UNE MAITRISE FINANCIERE ET TECHNIQUE. 
Author: Deschanels, Jean Louis; Lavedrine, Pierre 
Corporate Source: Aerospatiale 

Source: RGE, Revue Generale de TElectricite n 6 Jun 1 982, Journ d'Etud 
SEE La Surete de Fonct dans les Syst Ind Autom, Gif-sur-Yvette, Fr, Jun 4 
1981 p 445-447 

Publication Year: 1 982 

CODEN: RGELAC ISSN: 0035-31 1 6 

Language: FRENCH 

Journal Announcement: 8305 

Abstract: Any automated system, whatever the functions it must perform 
during all its life, presents risks of failure and may even be the origin 
of serious accidents or " dreaded events " , All these risks may cause 
significant financial losses. So, they must be allowed for as early as at 
the system stage. Though an objective of the security scheme is to optimize 
the system for a tradeoff between performance capabilities and costs, the 
objective of security entails defining the list of " dreaded events " , 
their level of acceptability and their probability of occurrence. The 
financial analysis of risks therefore consits in searching for the cost 
figure for residual losses due to any prospective accident occurring to the 
system in order to accept the whole risk of these losses or to define 
measures that permit financial control of these losses, either by a 
preventive policy, a protection policy or an insurance policy. In French. 



14/7/21 (Item 4 from file: 8) 
DIALOG(R)File 8:Ei Compendex(R) 
(c) 2000 Engineering Info. Inc. All rts. reserv, 

01 1 65036 E.I. Monthly No: EI821 21 1 1 968 E.I. Yearly No: EI82085955 
Title: MANAGING RISKS PART OF SUCCESS, 
Author: Zier, Robert E. 

Corporate Source: MITRE Corp, McLean, Va, USA 

Source: Solid Wastes Management v 25 n 5 May 1982 p 64-66, 68, 106 

Publication Year: 1 982 

CODEN: SWAMDJ 

Language: ENGLISH 

Journal Announcement: 8212 

Abstract: Resource recovery systems involve risks, not only on the part 
of private industry but on the part of government as well. The risk of 
exposure to financial loss results from a variety of potential causes such 
as failure of the technology, unanticipated environmental requirements and 
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the occurrence of natural disasters which can damage the facility or shut 
it down. Learning to handle these risks is essential to a successful 
project. For resource recovery projects, the occurrence of a risk can 
result in two undesirable outcomes for a municipal government: the cost of 
the system, or at least the cost of solid waste disposal, can exceed an 
acceptable figure; or needed capacity for solid waste disposal becomes 
unavailable. Identification of risk pinpoints the possible causes of 
undesirable outcomes and the corresponding probability of occurrence. Risk 
reduction is achieved by reducing either the probability of the occurrence 
of a risk or the impact of an undesirable outcome. Risk allocation Is 
assigning responsibility for controlling the cause and accepting 
responsibility for possible undesirable outcomes. The article discusses 
private sector risks, debt responsibility, risk sharing, and other aspects 
of the subject. 



14/7/24 (Item 3 from file: 34) 
DIALOG(R)File 34:SciSearch(R) Cited Ref Sci 
(c) 2000 Inst for Sci Info. All rts. reserv. 

07559345 Genuine Article*: 1 81 FE Number of References: 1 1 6 
Title: Risk and dangerousness 
Author(s): Buchanan A (REPRINT) 

Corporate Source: INST PSYCHIAT,DEFT FORENS PSYCHIAT, DE CRESPIGNY 

PK/LONDON SE5 8AF//ENGLAND/ (REPRINT) 
Journal: PSYCHOLOGICAL MEDICINE, 1999, V29, N2 (MAR), P465-473 
ISSN: 0033-291 7 Publication date: 1 9990300 

Publisher: CAMBRIDGE UNIV PRESS, 40 WEST 20TH STREET, NEW YORK, NY 

10011-4211 
Language: English Document Type: REVIEW 

Abstract: The task of improving the ability of clinicians to predict which 
of their patients will be violent has come to be seen as one of 
establishing the relative merits of actuarial and clinical prediction. 
The meaning of these terms is unclear. 'Clinicar is usually defined by 
exclusion, that is, as something other than actuarial. The term 
'actuariar is often used to refer to the techniques of risk prediction 
in financial services. In the psychiatric and psychological literature 
relating to the assessment of dangerousness, three further meanings 
have emerged. That whereby actuarial refers to any mathematical means 
of combining Information is the most widely accepted. Whichever 
definition is employed, the conclusion of most reviews has been that 
the future is actuarial. It is argued here that, while mathematical 
approaches have been successful in showing that risk factors for 
violence in the general population apply also to the mentally 
disordered, important questions remain unanswered. Mathematical methods 
address only one form of probability, that which arises from chance. A 
development of another form of probability, that which ar ises from 
causes, offers the prospect of improved risk assessment in psychiatry. 
It also offers a definition of clinical prediction that is not based on 
exclusion. 



14/7/40 (Item 4 from file: 94) 
DIALOG(R)File 94:JICST-EPIus 

(c)2000 Japan Science and Tech Corp(JST). All rts. reserv, 

00883101 JICST ACCESSION NUMBER: 89A0243178 FILE SEGMENT: JICST-E 
Development of corporate risk analyzing system based on financial data, 
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YOSHIDA TOSHIHIRO (1); TAKAMATSU KOSUMO (1) 
(1) Mitsubishi Res. Inst. Inc. 

Mitsubishi Sogo Kenkyujo Shoho(Journal of Mitsubishi Research Institute), 

1988, N0.15, PAGE.90-113, FIG.18, TBLIO, REF.16 
JOURNAL NUMBER: S0866AAY ISSN NO: 0287-2129 
UNIVERSAL DECIMAL CLASSIFICATION: 657.44 681.3:007.51 
LANGUAGE: Japanese COUNTRY OF PUBLICATION: Japan 

DOCUMENT TYPE: Journal 
ARTICLE TYPE: Original paper 
MEDIA TYPE: Printed Publication 

ABSTRACT: Quantitative analysis of financial statements has been playing a 
critical role in the field of corporate assessment for nearly a 
century, mainly because of its high objectivity. In recent years, 
advances in computer technologies have made software packages for 
advanced statistical analysis widely available and have also enabled 
computers to do qualitative reasoning. This stimulates the 
systematization of financial analysis. Meanwhile, drastic changes in 
management environment after the Oil Crisis in 1 973 have caused an 
increase in various kinds of risk in both private and public sectors. 
This problem has made the industries keenly aware of the importance of 
risk management. Considering these situations, we started a project 
which aims at examining financial analysis in the framework of risk 
management, proposing a conceptual design of a supporting system for 
risk management, and developing a prototype system. The designed system 
consists of two parts: (1) the statistical analysis component that 
detects something unusual from the movement of some pre-defined 
monitoring indices, and (2) the expert system component that Infers 
reasons of the anomaly and suggests measures to cope with the 
situation. The system is characterized by its unique hybrid structure 
combining a conventional statistical method and emerging Artificial 
Intelligence methods. In spite of a fairly limited knowledge and 
functon covered by the prototype, this system seems to have a quite 
effective analyzing ability. In the case of RICCAR Corporation, one of 
the largest sewing machine makers In Japan, the system Indicated 
defects In sales activity of the firm four years before its bankrupcy. 
A succeeding project with a goal of developing an enhanced knowledge 
based and practical system Is now underway. (author abst.) 

?t 514/7/6,8,32,33,36,62,63,64,65,73,74,92 

1 4/7/6 (Item 6 from file: 2) 
DIALOG(R)File 2:INSPEC 

(c) 2000 Institution of Electrical Engineers. All rts. reserv. 

4995552 INSPEC Abstract Number: B9508-01 60-009 
Title: From determinism to probabilism 
Author(s): van Otterloo, R.W. 
Author Affiliation: N.V. KEMA Arnhem, Netherlands 
Journal: Microelectronics and Reliability vol.35, no.9-10 p.l 357-62 
Publication Date: Sept.-Oct. 1 995 Country of Publication: UK 
CODEN: MCRLAS ISSN: 0026-2714 

U.S. Copyright Clearance Center Code: 0026-271 4/95/$9.50+.00 
Language: English Document Type: Journal Paper (JP) 
Treatment: General, Review (G) 

Abstract: Probabilistic risk assessment (PRA) is to a certain extent a 
luxury. It Is a luxury that is applied In the margin of large projects 
where decisions making is very important and involves the safety of large 
groups of people or the possibility of causing environmental damage or a 
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large loss of financial investments. The field of application of PRA is 
therefore restricted. There is just one way of enlarging the application 
field and that is by further automation for the PRA techniques. Further 
automation makes PRA's cheaper and therewith will small margins appear 
large enough. (0 Refs) 
Copyright 1995, lEE 



14/7/8 (Item 8 from file: 2) 
DIALOG(R)File 2:INSPEC 

(c) 2000 Institution of Electrical Engineers. All rts. reserv, 

4931938 INSPEC Abstract Number: C9506-1 290D-006 
Title: Martingale analysis for assets with discontinuous returns 
Author(s): Bardhan, I.; Chao, X. 

Author Affiliation: Goldman Sachs & Co., New York, NY, USA 
Journal: Mathematics of Operations Research vol.ZO, no.1 p.243-56 
Publication Date: Feb. 1 995 Country of Publication: USA 
CODEN: MOREDQ ISSN: 0364-765X 

U.S. Copyright Clearance Center Code: 0364-765X/95/2001 /0243/$01 .25 
Language: English Document Type: Journal Paper (JP) 
Treatment: Theoretical (T) 

Abstract: The equivalent martingale measure approach Is applied to a 
financial market subject to jump-diffusion uncertainty. The uncertainty in 
the market is caused by a multidimensional Brownlan motion process and a 
multidimensional point process of Jumps admitting stochastic intensity. 
Under a boundedness condition on the relative risk premium on jumps, an 
equivalent risk-neutral probability measure is identified and is used to 
construct hedging portfolios for consumption processes and contingent 
claims. These are then applied to problems of utility maximization. (18 

Refs) 

Copyright 1995, lEE 



14/7/32 (Item 1 1 from file: 34) 
DIALOG(R)File 34:SciSearch(R) Cited Ref Sci 
(c) 2000 Inst for Sci Info. All rts. reserv. 

05074634 Genuine Article#: TN531 Number of References: 37 

Title: THE PRODUCTIVITY OF FINANCIAL INTERMEDIATION AND THE TECHNOLOGY OF 

FINANCIAL PRODUCT MANAGEMENT 
Author(s): HOLMER MR; ZENIOS SA 

Corporate Source: HR&A INC/WASHINGTON//DC/00000; UNIV CYPRUS,SCH ECON & 

MANAGEMENT/NICOSIA//CYPRUS/ 
Journal: OPERATIONS RESEARCH, 1995, V43, N6 (NOV-DEC), P970-982 
ISSN: 0030-364X 

Language: ENGLISH Document Type: ARTICLE 

Abstract: Financial intermediaries-banks thrifts, and life insurance 

companies-have experienced low productivity over the last decade or 

two. Low productivity has manifested itself as a declining market share 

of their products relative to capital market assets. In some cases, low 

productivity caused a failure to meet contractual obligations embodied 

in their financial products. These failures resulted in customer 

losses, and/or taxpayer losses when failed intermediaries were 

guaranteed by government agencies. This productivity problem has been 

analyzed mostly from an economic science perspective, by R. C. Merton 
(1 990) and Z. Bodie (1 990). The focus of the economic analysis is the 

improvement of regulatory measures for intermediaries whose financial 
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products are guaranteed by government agencies. In this paper we take a 
management science perspective by focusing on the technology of 
financial product management. An assessment of current technologies 
finds that their use can leave financial intermediaries exposed to 
substantial risks. An improved technology-integrated product management 
(IPM)-is suggested that enables intermediaries to increase 
productivity. Therefore, they can respond more effectively to market 
pressures from competing capital market assets and to regulatory 
pressures from government agencies. Technical and organizational 
aspects of integrated product management are described, and its 
application to three examples is discussed. The problem outlined here 
presents a major challenge to management scientists. It is an example 
of the service-sector applications that A. Geoffrion (1992) addressed 
in his 1 991 Omega Rho lecture. 
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Abstract: The relationship between financial statement information and 
security market information has been the subject of intensive 
theoretical and empirical research. The bulk of empirical studies 
consider this relationship without an explicit consideration of 
causality. In the present study we put forward some novel results 
concerning the direction of causality between these two information 
sets. Our results indicate that the accrual information in financial 
statements has some projective value with respect to future reactions 
on the security markets. 
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Abstract: Three focal elements of knowledge-based system design are (i) 
acquiring information from an expert, (ii) representing the information 
in a system-usable form, and (iii) using the information to draw 
inferences about specific problem instances. In the artificial 
intelligence (Al) literature, the first element is referred to as 
knowledge acquisition, while the second and third are embodied in a 
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system's knowledge base and inference engine, respectively. Al, 
however, is not alone in its concern for these issues. Researchers in 
several of the statistical decision sciences, notably decision analysis 
(DA), have also investigated them. This paper discusses the use of 
belief networks-a formalism that lies somewhere between Al and DA-as an 
overall framework for knowledge-based systems. Unlike previous work, 
which has concentrated on either the networks' mathematical properties 
or on their implementation as a specific system, this paper is oriented 
towards the concerns of general system design. Concrete examples are 
drawn from one medical system (Pathfinder) and from one financial 
system (ARC01 ), and in particular, from a consideration of their 
similarities and differences. The design principles abstracted from 
these systems suggests a powerful, coherent design philosophy guided by 
the simple thought: form follows function. 
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The word ^ ^complex" in the title of the book under review has nothing to 
do with the complex numbers. Day distinguishes simple models from complex 
ones and simple dynamics from complex dynamics. Simple (economic) models 
with simple dynamics are the type of model studied in (economic) textbooks. 
Governments often base their policies on predictions of a central planning 
board. These CPBs often base their predictions on complex models, but with 
simple dynamics. Predictions have a 'Mocal" character, as they are made 
only for a small neighbourhood of the relevant parameters. As such, the 
functions describing the developments of the economy can be chosen to be 
linear. Complex dynamics, such as chaotic dynamics, are only possible with 
nonlinear functions. Day's book deals with simple models, but with complex 
dynamics. 

I have written several reviews on chaotic dynamics, although I have 
never used it in my own research. Every time I have written such a review, 
I have pointed out that the occurrence of chaotic dynamics in economic 
models should be based on economic assumptions. The wrong way to deal with 
it is to make some unrealistic, ad hoc assumptions that lead to 
well-understood models in other sciences. Every time I wrote that, I had in 
mind a special model of the author [Amer. Econ. Rev. 72 (1982), no. 3, 
406-414], I must admit when I started reviewing this book I was 
prejudiced. Reading the book has taken part, but not all, of the prejudice 
away. 
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There are two pitfalls for authors of books on mathematical economics in 
general, and in particular, for those of books on complex economic 
dynamics. Either the mathematics Is too difficult for the average 
economist, or the economics is too difficult for the average mathematician. 
This book, the first of two volumes, deals with economic applications only 
in the last part. The economic examples of the last part are an interesting 
illustration of the mathematical theory of the second part. It is a pity 
that this part is rather short. More applications are promised In the 
second volume, which has not yet been published. Day's book will not be too 
difficult for the average economist who has some familiarity with, but not 
necessarily an advanced knowledge of, mathematical modelling. Proofs are 
given only in the simple cases, sketches of proofs for the slightly more 
difficult ones and only a statement of the results for the more complicated 
cases. 

Doubtless Day saw, at an early date, the possibilities of applying 
nonlinear and chaotic dynamics to economics. He has since then been working 
on the subject and has stimulated others to do the same. It is a good thing 
he has written about his experience in a book. The book under review 
contains three parts. The first part Is a philosophical and methodological 
introduction. There are a striking number of references to the literature 
on physics, as if Day regrets he studied economics. 

The second part of the book deals with the mathematics of discrete, 
one-dimensional (semi)dynamical systems. All the familiar subjects: 
stability, instability, cycles, chaos, bifurcation, itineraries, 
ergodicity, the Frobenius-Perron operator, the Schwarzian derivative, etc., 
are there. The section on " "multiple phase'* and " "statistical" dynamics 
is rather extensive. In the former, the transition map has different 
analytical expressions on different regions of the domain. Piecewise linear 
(affine) maps are an example of these. This part Is a fine introduction to 
the subject; however, to those interested, I would recommend P. Collet and 
J.-P. Eckmann [ Iterated maps on the interval as dynamical systems, 
Birkhauser, Boston, MA, 1980; MR 82j:58078] which covers most of the 
subjects dealt with in this part of Day's book as well as giving all 
proofs. 

The third and last part of this volume deals with economics. It was in 
reading this part that from time to time the feeling that caused my 
prepossession popped up again. The first chapter of this part deals with 
Walras' tatonnement. Stability of the competitive system was the topic In 
the fifties. The Sonneschein, Mantel and Debreu theorem shows that all 
continuous, homogeneous functions of degree zero that satisfy Walras' law 
can be the excess demand function of an economy. As such, " "pathological 
examples" cannot be excluded from economic theory. Moreover, the 
tatonnement process is not a realistic presentation of reality. Day shows 
that even in a very simple way, chaos can be Introduced In the tatonnement 
model. 

The second chapter of this part Introduces a mediator between suppliers 
and demanders in the tatonnement process. With this mediator, the model 
becomes more natural. Further, the stock market Is explained by this model. 
It leads to the interesting, but not new, conclusion that theoretical data 
generated by a deterministic process cannot be distinguished from actual 
data generated by a stationary stochastic process. The final chapter 
introduces financial feedbacks to the tatonnement model, which Is treated 
as a "cobweb' model in this chapter. 
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Contents:\ M. Beckmann, W. Eichhorn, W. Krelle, 0. Opitz and B. Rauhut, 
Publikationen und Editionen von Rudolf Henn [Publications and editions by 
Rudolf Henn] (pp.\ v--ix). 

Section 1, Mathematical foundations and mathematical theory of 
economics:\ Karl-Heinz Elster and Diethard Pallaschke, Strukturell stabile 
Vektorfelder als allgemeine Konjunkturmodelle im Sinne von H, Rose 
[Structurally stable vector fields as general models of business cycles in 
the sense of H. Rose] (pp. 3--1 1); Werner Hildenbrand, Zur Analyse von 
Gleichgewichten in grossen Okonomien [On the analysis of equilibria in 
large economies] (pp. 12—22); Susanne Kemmerich and Michael M, Richter, 
Bemerkungen uber die Automorphismengruppe von homogenen Boole'schen 
Algebren [Remarks on the automorphism group of homogeneous Boolean 
algebras] (pp. 23—28); Heinz Konig, On the main theorems of superconvex 
analysis (pp. 29-34); Peter Kosmol, Uber die sukzessive Wahl des kurzesten 
Weges [On the successive choice of the shortest path] (pp. 35-42); Olaf 
Krafft and Martin Schaefer, A Gale-Ryser theorem for incidence-structures 
(pp. 43-48); Marco Lehmann-Waffenschmidt, Bounding the price space ${\bf 
R}\sp n\sb +$ by a collar-preserving homeomorphism (pp. 49-62); Joachim 
Rosenmuller, An algorithm for the construction of homogeneous games (pp. 
63-74); Norbert Schmitz, Superspiele mit Vertrauensbildung [Supergames 
with confidence formation] (pp. 75—88); Walter Vogel, Sequential 
approximation with errors in normed linear spaces (pp. 89—97). 

Section 2, Optimization:\ Michael Bastian, Zur Losgrossenbestimmung in 
mehrstufigen Mehrgutersystemen mit allgemeiner Fixkostenstruktur bei 
stationarer Nachfrage [On the determination of lot sizes in multilevel 
multiproduct systems with a general fixed cost structure in the case of 
stationary demand] (pp. 101-116); Konrad Boenchendorf, Eine Klasse von 
Facetten fur ein gemischt-ganzzahliges Programm mit Fixkosten [A class of 
facets for a mixed integer program with fixed costs] (pp. 117-125); 
Ulrich Eckhardt and Thomas Kemmann, Ein Verfahren zur Losung des Least 
Distance Problems [A method for the solution of the least distance problem] 
(pp. 126-138); Reiner Horst and Le Van Dien, A solution concept for a very 
general class of decision problems (pp. 139-149); Franz J. Radermacher, 
Characterization of the critical posets in two-machine unit time 
scheduling (pp. 1 50-1 58); Werner Rothengatter and Michael Wiedemann, 
Zuglauf und Zugbildung in Schienennetzen — Ein Anwendungsfall fur die 
konvexe Optimierung [Train scheduling and train formation in railway 
networks — a case study for convex optimization] (pp. 159—172); Richard 
Vahrenkamp, Optimale Travelling-Salesman-Touren [Optimal traveling 
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salesman tours] (pp. 173—183). 

Section 3. Stochastics and network planning:\ Martin J. Beckmann, 
Subjective probability in adaptive decision processes (pp. 1 87—191); Kuno 
Egle and Szaniszio Fenyi, Stochastische Inversion von Leontief-Matrizen 
[Stochastic inversion of Leontief matrices] (pp. 192-205); Ortwin Emrich, 
Verfahren zur Bestimmung optlmaler Stoppmengen [Procedures for determining 
optimal stopping sets] (pp. 206-215); Kurt Marti, Optimally controlled 
semi-stochastic approximation procedures (pp. 216-230); Martin Morlock 
and Klaus Neumann, Stochastic single-machine scheduling to minimize the 
sum of expected weighted completion times subject to OR precedence 
constraints (pp. 231-243); Henning Paul, Dynamische Zeitplanung in der 
Netzplantechnik [Dynamic scheduling in network planning] (pp. 244-250). 

Section 4. Statistics and econometrics:\ Joachim H. Ahrens, A comparison 
of hypergeometric distributions with corresponding binomial distributions 
(pp. 253-265); Wolfgang Domschke and Andreas Drexl, Planung der 
Stichprobeninventur mit Hilfe eines probabilistischen Austauschverfahrens 
[Planning of a sample inventory with the help of a probabilistic exchange 
procedure] (pp.\ 266-275); Bernd Goldstein and Volker Steinmetz, Zur 
Prazisierung des Zusammenhangs zwischen Bereichsschatzfunktionen und 
Parametertests [On the precision of the connection between domain 
estimators and parameter tests] (pp. 276-285); Joachim Hartung and 
Karl-Heinz Klosener, Zur Bestimmung der Prazision von Messverfahren [On 
the determination of the precision of measurement procedures] (pp. 
286-296); Jochen Hulsmann, $L\sb 1$-norm Schatzung der Verteilungsfunktlon 
[$L\sb 1$-norm estimation of the distribution function] (pp. 297-301); 
Otto Opitz, Optimale Skalierung qualitativer Oaten [Optimal scaling of 
qualitative data] (pp. 302-316); Burkhard Rauhut, The modelling of outlier 
situations (pp. 317-324); Martin Schader and Friedrich Schmid, On the 
$\chi\sp 2$ test for a negative binomial distribution (pp. 325-330); 
Bernd Schips, Grenzen klassischer Interferenzkonzepte im okonometrischen 
Modellbau [Limits of classical concepts of interference in the construction 
of econometric models] (pp. 331-340); Hans Schneeweiss and Horst 
Witschel, Small sample properties of estimators in a linear relationship 
with trend — a Monte Carlo study (pp. 341—352). 

Section 5. Applied computer science:\ Wolfgang H. Janko and Reinhard 
Feurer, Eine Studie zur Beurteilung der sprachlichen Eignung der 
Programmiersprachen BASIC, PASCAL, APL, APL2, LISP und PROLOG zur 
Programmierung regelbasierter Systeme [A study to assess the linguistic 
suitability of the programming languages BASIC, PASCAL, APL, APL2, LISP 
and PROLOG for the programming of rule-based systems] (pp. 355—364); 
Hartmut Noltemeier, Voronoi trees (pp. 365—371). 

Section 6. Microeconomlcs:\ Horst Albach, Bernd Franke and Michael 
Werhahn, Bemerkungen zur Theorie der Investitionsketten [Remarks on the 
theory of investment chains] (pp.\ 375—380); Gunter Bamberg and Franz 
Baur, Commodity futures markets and the level of production (pp. 
381-395); Wolfgang Gaul, Zum Einsatz von Datenanalysemethoden in der 
Marktforschung [The application of data analysis methods in market 
research] (pp. 396-408); Hermann GoppI, Neue Instrumente zum Management 
finanzieller Risiken [New instruments for the management of financial 
risks] (pp. 409-415); Gerhard Hieber, Uberlegungen zum 
Informationsmanagement im Bankgeschaft [Considerations on information 
management in banking] (pp. 416-419); Peter Kischka, Risikoeinschatzung 
und optimaler Versicherungsschutz: einige Anmerkungen [Risk evaluation and 
optimal insurance protection: some remarks] (pp. 420-428); Arnold Kremer, 
Der Einfluss der Zinsstruktur auf die Ertragsentwicklung einer 
genossenschaftlichen Zentralbank — Eine okonometrlsche Analyse [The 
influence of the interest structure on the profit development of a 
cooperative central bank — an econometric analysis] (pp. 429-444); Egon 
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Kremer, Finanzinnovationen und ihre Akzeptanz auf den internationalen 
Markten [Financial innovations and their acceptance In international 
markets] (pp. 445--460); Reimer Schmidt, Zur jungeren Entwicklung der 
Versicherungswissenschaften [On the recent development of the insurance 
sciences] (pp. 461-475); Robert Schwebler, Konjunkturprognosen und 
Versicherungswirtschaft [Economic predictions and the economics of 
insurance] (pp. 476-486); Franz Steffens, Technische Optlmierung und 
Nettobedarfsbildung in isotonen Input-Output-Systemen [Technical 
optimization and formation of net needs in Isotonic input-output systems] 
(pp. 487--503); Klaus Zoller, Koordination und Wirtschaftllchkelt In 
Mehrprodukt-Lagern [Coordination and efficiency in multiproduct 
warehouses] (pp. 504—512). 

Section 7. Macroeconomics and public economlcs:\ Klaus Ballarini and 
Georg Bol, Investition und glelchgewlchtlges Wachstum bel konstantem 
Arbeitsangebot [Investment and equilibrium growth with constant employment 
opportunity] (pp. 515-522); Wolfgang Elchhorn and Frank Stehling, EIne 
Bemerkung zur Verteilungsneutralitat der produktivitatsorlentlerten 
Lohnpolitik [A remark on the distribution neutrality of 
productivity-oriented wage policy] (pp. 523—532); Rolf H. Funck, Jan S. 
KowalskI and Reiner Koblo, New technology, innovative activities, and the 
German city system (pp. 533—543); Klaus Hellwig, Die Erhaltung der 
volkswirtschaftlichen Leistungsfahigkelt [The preservation of the 
productivity of the national economy] (pp.\ 544—548); Alexander Karmann, 
Karl-Heinz Ketterer and Gholamreza Nakhaelzadeh, Zu den Beziehungen 
zwischen Geldmenge und Gesamtnachfrage — einige Ergebnisse einer 
Kausalltatsuntersuchung [On the relations between monetary supply and total 
demand — some results of a causality investigation] (pp. 549—557); Hartmut 
Kogelschatz, Zur Preisentwicklung erschopfbarer Ressourcen [On the price 
development of exhaustible resources] (pp. 558—565); Wilhelm Krelle, 
Structural change induced by foreign trade. What are the advantages for 
the country? (pp. 566-598); Gerhard Seller, Kommunale Finanzplanung als 
Aufgabe des Operations Research [Communal financial planning as a task of 
operations research] (pp.\ 599—609); Klaus Spremann, Zur Okonomie des 
Heldentums [On the economics of heroism] (pp. 610—617); Armin 
Schmiedeberg and Joachim Voeller, Formale und inhaltliche Glelchheit — zur 
Antinomic verfassungsrechtlicher Grundprinzlpien [Formal and intuitive 
equality — on the antinomy of fundamental constitutional principles] (pp. 
618—634); Manfred Zach, Neue Technologien und Beschaftlgung. 1st 
Technologiepolitik beschaftigungsfeindlich? [New technologies and 
employment. Is technology policy Inimical to employment?] (pp. 635-642). 

Section 8. Economic science and praxis:\ Achim Zink, 
Wirtschaftswissenschaft und Praxis [Economic science and praxis] (pp. 
645-648). 
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Statistics. A Journal of Theoretical and Applied Statistics, 1 986, 1 7 
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Subfile: MR (Mathematical Reviews) AMS 
Abstract Length: SHORT (9 lines) 

Summary: ^'We formulate a general statistical decision problem for 
reliability investigations and its specification for reliability tests with 
test period terminated by time and failures. The risk function includes the 
experimental costs as well as the expected financial expenses caused by the 
decisions, interpreted as "rejection' and "acceptance', which are to be 
made when the test result is given. Under the assumption that the lifetime 
distribution of the items to be tested Is exponential, we evaluate the 
Bayes risk for a wide class of prior distributions." 
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14/7/65 (Item 7 from file: 239) 
DIALOG(R)File 239:Mathsci(R) 

(c) 2000 American Mathematical Society. All rts. reserv. 
02522175 MR 81g#62147 

Nonparametric estimation of functions in a model of competing risks from 
incomplete longitudinal data. 
Mode, Charles J. 
Math. Biosci. 

Mathematical Biosciences. An International Journal, 1979, 45, no. 
1-2, 1-20. ISSN: 0025-5564 CODEN: MABIAR 
Language: English 
Document Type: Journal 
Journal Announcement: 1 1 22 
Subfile: MR (Mathematical Reviews) AMS 
Abstract Length: LONG (37 lines) 

The classical competing risk problem is concerned with assessing 
the distribution of the random variable T{in}(0, {infin}), representing 
time to death (or breakdown, etc.), while at the same time keeping track 
of l=(1, ... ,k), representing cause of death. Thus, let G(t)=P(T <= t) 
and G {sub}i(t)=P(T <= t, l=i); the states 1 , ... ,k are assumed to be 
mutually exclusive and exhaustive, so that G(t)=G{sub}1 (t)+ 
...+G{sub}k(t). The author studies the problem of estimating the 
G{sub}i under the assumption that T has a discrete distribution and that 
each item (or individual) under study has a preassigned "cut-off" time 
at which it is censored away, if it has not yet broken down. In this 
framework, the estimation problem reduces to a combination of standard 
multinomial situations. Least-squares and maximum likelihood solutions as 
well as their large-sample properties are then fairly standard (although 
the author has chosen to provide a rather elaborate treatment). The 
paper seems somewhat out of touch with the current strong interest in the 
competing risk problem. A comprehensive study of nonparametric estimation 
of the G{sub}i in continuous time under quite arbitrary kinds of censorship 
was given by 0. Aalen (Ann. Statist. 6 (1978), 534 - 545; MR 57 #17987). 
One interpretation of the G{sub}i is that they are derived from postulated 
independent "latent failure times" T{sub}i, so that T=mln(T{subp , ... ,T 
{sub}k) and G{sub}i(t)=P(T{sub}i <= T{sub}j, j{neq}i, T{sub}i <= t). This 
approach is summarized in the recent monograph by H. A. David and M. L. 
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Moeschberger (The theory of competing risks, Griffin, London, 1 978). 
However, D. R. Cox, in a review of David and Moeschberger's book 
(Internat. Statist. Rev. 48 (1980), no. 2, 234 - 235), as well as, among 
others, R. L Prentice et al. (Biometrics 34 (1978), no. 4, 541 - 554), 
pointed out that the hazard rates lambda 

{sub}i(t)=lim{sub}(h{arrd}0)(1/h)P(T{sub}i <= t+h {vert} T > t) are well 
defined without the independence assumption and therefore better suited 
as fundamental objects of the statistical inference. In the paper under 
review these points are only briefly and incompletely raised. 
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TEXT: 

An integrated knowledge-management product does more than just 
capture expertise-it becomes a way of doing business. 

Intraspect Knowledge Server 2.0 and Livelink 8.0 provide almost 
everything. 



Knowledge management is more of a concept, a way of doing business, 
than it is a type of product. That concept is made possible by four basic 
processes: gathering, organizing, refining, and disseminating. Gathering 
brings information and data into the system. Organizing associates items 
with subjects, giving them context. Refining adds value by discovering 
relationships, performing synthesis, and abstracting. Disseminating gets 
knowledge to the people who can use it in a form that makes it usable. I 
Knowledge solutions require all four processes, though they don't require 
that all of them be accomplished through information technology. By 
integrating the four processes, the solution should not only provide the 
data on which decisions are based, but also put it in context and 
distribute it to a large set of users. I The most frequently missing 
connection in past knowledge-management solutions has been between the 
refining process and the other three factors. Most "knowledge" offerings 
come from either the specialized refining-tools companies (producers of 
online analytical processing clients or data mining front ends) or 
companies focused on delivery or search. Knowing which processes a 
knowledge- management product supports will help you figure out what other 
ingredients you'll need. I The two products we looked at, Intraspect 
Software Inc.'s Intraspect Knowledge Server 2.0 and Open Text Corp.'s 
Livelink Intranet 8.0, cover three of the four knowledge-management 
processes with their intrinsic technology, they leave out refinement but 
enable some human refinement processes. Both are among the most fully 
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evolved knowledge-management platforms on the market. Both will be 
appropriate in their current state for many companies looking for 
knowledge-management products. 

Intraspect Knowledge Server 2,0 

More than other knowledge-management technologies, Intraspect 
Knowledge Server is designed to integrate with users* business processes, 
letting them capture knowledge in the context in which the knowledge was 
used. The system provides a turnkey collaborative environment that 
provides organizations an easy way to access, track, and share enterprise 
data. It also lets users easily add knowledge to their organization's 
"group memory" as part of their usual work processes. 

The company is aiming its product at large businesses, focusing on 
the telecommunications, pharmaceutical and health, financial services, and 
manufacturing industries. Intraspect sells its products directly and 
through resellers and system integrators. 

Intraspect is a collaborative environment for collecting, finding, 
and reusing information in an organization, letting knowledge workers 
create and maintain a knowledge store or "group memory" of their work. One 
of Intraspect's major business applications is the "collaborative 
corporate portal," a personalized interface to the information used by 
people in an organization, organized by its context of use. In a typical 
portal application, users subscribe to the information they need, working 
collaboratively with that information within business processes. 

A major differentiator for Intraspect is its collaborative approach 
to content management. An organization's knowledge workers create the 
hierarchies and add the content-unlike many other systems, which assign 
these responsibilities to an administrator or knowledge manager. With 
Intraspect, users with the proper permission can create their own 
organizational hierarchy (or context) within the group memory and add the 
content they deem appropriate. Groups can also establish their own set of 
contexts. Organizations can choose from a wide variety of 
knowledge-management applications, ranging from centrally controlled 
catalog servers to highly distributed group memories. 

The system is also very flexible, letting users subscribe to only 
the portions of the group memory that are vital to their tasks. Users can 
subscribe to specific searches (for example, searches that identify 
business topics), discussions, messages, documents, Web pages , comments, 
and folders. 

The system is both Windows- and Web browser-compliant, and will be 
easy to learn for users who are comfortable in either environment. 
Moreover, administrators can perform all their management tasks remotely, 
via the Web. 

Intraspect is tightly integrated with the desktop environment, 
including applications such as Microsoft Office and most E-mail systems. 
Users can capture any digital content, whether from business- intelligence 
systems, data warehouses, enterprise resource planning systems, or 
customer-relationship management systems, as well as file systems, Web 
pages, or E-mail messages with attachments. 



However, the system doesn't make full use of the operating system's 
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capabilities. For example, Intraspect doesn't leverage Windows NT 
security, user and group definitions, or administration utilities. 

Intraspect has a role-based security model that has been designed 
for collaborative use. Any object, from cabinets and folders to individual 
documents, can be restricted. Users and groups can be assigned to reader, 
contributor, organizer, editor, and publisher roles. This lets an 
organization create a variety of sharing environments in the group memory, 
from strict propose-review-publish workflow to a collaborative work area, 
giving members of a group free access to each other's work. Anonymous 
(unauthenticated) Web access can also be turned on for any subset of 
objects in the system. 

Intraspect has tight integration with HTTP and the Simple Mail 
Transfer Protocol. Intraspect exposes SMTP, Java, and HTML-level 
application programming interfaces for integration with E-mail systems and 
custom client development. The HTML API provides several prebuilt 
templates for rapid customization. 

Intraspect is designed for wide participation in an organization, 
and any user can contribute information to the group memory. Users can 
create folder hierarchies that make sense to them, then add to the group 
memory by dragging and dropping files from the Windows Explorer and URLs 
from a Web browser, or using a browser-based HTML form. Users can also 
"mail in" documents to a personal "collection bin" or directly to a 
folder. Users gathering information without knowing where it should go can 
contribute to their collection bin and move it into the appropriate folder 
later. This translates into more thorough collection and more accurate 
filing, both of which advance the cause of knowledge management. 

Once documents are added to the group memory, users can 
automatically be notified of the update, either through E-mail or 
Intraspect's personalized Web interface, 

A simple document upload template lets users profile information 
when adding it to the group memory. The template automatically captures 
the author's name and the date the document was created, and lets users 
add a title and description of the new document. If the document is 
modified, the system tracks the name of the user who modified it and the 
date that it was modified. You can use HTML to modify templates, adding 
more fields or changing the look and feel of the interface itself. Fields 
added to the metadata of the document are saved in the repository and are 
searchable using the built-in description parameter. 

A very important feature: A single file in the group memory can 
exist in multiple contexts. When users select the page, they can see the 
multiple contexts in which It has been used, which may be helpful for 
additional information about the subject and to understand better who in 
an organization knows about the subject and in what ways. 

Intraspect's user-control level is extremely powerful and will work 
for companies in which users are sufficiently disciplined to provide 
logical structure to their hierarchies. However, companies that have ad 
hoc managers run the risk of ending up with a highly unorganized and 
chaotic containment model. For them, Intraspect recommends using the 
access-control capabilities to give control over the top of the hierarchy 
to information specialists who focus on organizing the information and 
creating semi-private workspaces for collaboration. 
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Features that continue to distinguish Intraspect from most other 
knowledge-management tools are its collaborative features and E-mail 
integration. Intraspect lets users create and participate in multithreaded 
discussions within a hierarchy. Discussion items are saved in the system 
in context. Users can create distribution lists for documents and 
discussions by selecting the user's E-mail address. In addition, users can 
create annotations over the Web. 

Collaboration can even include people outside the organization. 
Since each folder has its own unique E-mail address, outside users with 
rights can send messages directly into Intraspect. This feature is highly 
valuable for extranet applications in which organizations want to capture 
messages from customers or suppliers. 

Livelink 8.0 

Open Text focuses on providing products for businesses and 
government offices that want to use intranets and extranets for 
collaborative processes. Livelink is Open Text's flagship Web product; it 
Includes document management, knowledge management, and workflow out of 
the box. Open Text sells Livelink directly and through channel partners. 

Open Text positions Livelink as a collaborative, enterprisewide 
knowledge- and document-management product for managing information, 
resources, processes, and time. 

Currently, Open Text partners with Kofax Image Products Inc. and 
Input Software Inc. to offer Kofax's Ascent Capture or Input Accel as a 
separate, integrated module for converting paper documents to TIFF graphic 
images. For managing time, Livelink offers OnTime, the descendant of the 
first scheduler built for Microsoft Windows. The most recent version is 
available as a fully integrated Livelink module , offering synchronization 
of its enterprise calendar with PalmPilot, Windows CE, Microsoft Exchange, 
Lotus Organizer, Outlook 97/98, Sidekick, Act, and Lotus Notes. 

Livelink offers a set of core services, including a document- 
management repository (for information contribution), workflow (for 
managing process collaboration), virtual team collaboration (for project 
teams), and full-text search and retrieval. It also offers several 
configurable modules, including Livelink Spider (for indexing external Web 
pages and providing "push" updates), Livelink Forms (for gathering user 
input and routing form data through workflows), Livelink Desktop Activator 
(for Open Document Management API support), Livelink Activators for SAP 
R/3 and Lotus Notes (for providing access to other enterprise systems), 
Livelink Change Agents (for centralized management of Web-site changes and 
notification via E-mail), and Livelink Prospectors (for creating 
personalized virtual research assistants). 

Livelink furnishes several searching protocols, including Boolean, 
proximity, thesaurus, soundex (provides a list of words that sound like 
the words in the query), stemming, and date range. Conceptual searching is 
not supported, however. 

Overall, Livelink is the most complete product we have seen in the 
constellation of knowledge-management offerings. Moreover, development 
efforts are under way to enhance its existing capabilities. 
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There is a serious downside, however. Many users will find the user 
interfaces confusing and difficult to understand and navigate. The natural 
tendency for more compound products to be more difficult to learn holds 
here. The different functions, while well-developed, operate 
inconsistently and give this powerful product a "cobbled together" feel. 

Livelink features one immensely practical innovation-It divides the 
user interface into three consoles, each with its own special view that 
corresponds very similarly to the way knowledge workers' daily tasks fall 
out. One view is for the enterprise and delivers "big picture" information 
everyone in the organization needs to see. The second interface is the 
project view, in which users spend time working on designated projects. 
With the appropriate permissions, a user can set up a project, designate 
its members* roles and responsibilities, build a workflow, and incorporate 
discussion groups. The third interface is the personal view, used for 
noncollaborative projects and tasks. All the functions of the project view 
are available in the personal view. 

This three-level model works well for user productivity, keeping 
focus on one area at a time. If areas are blended, as with most products, 
many employees dissipate work energy shifting from one to the other. 
All-in-one consoles make for good marketing, but they undermine employee 
effectiveness. 

Users can search and retrieve office documents, discussions, 
workflows, tasks, and projects. Search protocols provide a variety of 
methods for conducting a search. Users can also combine attribute- based 
searching with full-text content. Livelink uses its own full- text search 
engine. All results are user-configurable and include relevancy ranking 
and auto summarization. 

Livelink uses a Web spider to index designated Web sites 
automatically and push updates to the subscribed user for immediate 
notification of changes to content. Users can receive notification through 
Livelink natively or through a MAPI-compliant E-mail system. Information 
retrieval also furnishes a variety of reports that can be presented in 
text, graphs, or tables. Only users with the appropriate permissions can 
view these reports. 

Livelink's Knowledge Library goes far beyond traditional document 
management, providing support for virtually any file type, as well as 
on-the-fly conversion to HTML for viewing, version control, and compound 
document support. In addition, the Knowledge Library is completely 
object-based. Nine levels of permissions exist on each object, with full 
audit trails for all document categories and attributes. 

Virtual Team Collaboration lets a project owner define the project 
overview, project outline, roles of each member of the virtual team, task 
lists, threaded discussion, news channels, and change agents. For more 
complex projects, the project owner can even create subprojects as a means 
of more granular control. 

Livelink supplies workflow based on Java, with drag-and-drop 
workflow definition maps, over the Web. Livelink's workflow features 
include subflows; dynamic routing; conditional, parallel, or serial 
routing; and rules-based triggers, such as milestones, loop-backs, and 
permissions. Livelink integrates with Adobe Systems' intelligent forms to 
assist in the routing process as well. 
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When retrieving documents, the user has the option to view the 
document in HTML or its native format. When viewing the document in its 
native format, the user must have the native application resident on the 
worl<station. Unfortunately, Livelink doesn't support the use of 
annotations. 

Livelink lets users search across multiple databases simultaneously 
and retrieve a single, unified hit list. Users, therefore, don't need to 
know where information is located, making the search experience less 
stressful. In addition, the user can control the look and feel of search 
results. They may opt for automatic summarization, relevancy ranking, or a 
simple hit list for display search results. 

LIvelink's drag-and-drop capabilities are impressive. A user can 
add documents to the Livelink Knowledge Library by opening Windows 
Explorer and dragging the document or documents to the Livelink Explorer. 
This feature is Ideal for users comfortable working in a tree-view 
environment. 

ODMA support lets users add documents to the Knowledge Library 
directly from Microsoft Word by selecting "File/Save As." The Livelink 
profile screen automatically opens for indexing to the Knowledge Library, 

Unfortunately, profiling E-mail messages into the Knowledge Library 
is a manual process. We were also disappointed that only one person and/or 
group can be assigned to a workflow task. The capabilities and graphical 
design of the workflow map is impressive, including map version control 
over the Web, and the ability to create reusable master workflow templates 
and to archive documents and their workflow instance upon completion of 
the workflow. But reusing these master templates Is not an intuitive 
process. 

The program doesn't highlight the hit results for full-text 
searches in the viewed document, making navigation laborious, especially 
in longer documents. The company says it's addressing this problem in the 
next release, Livelink 8,1,1. Finally, while Livelink offers more features 
and capabilities than similar products, the overall layout and 
presentation is inconsistent from one page to the next. Simplifying user 
interfaces and making each page both consistent and similar to a typical 
Web-page layout would greatly increase Livelink's ease of use. 

Open Text's offering is stunning in Its integration with workflow 
and document management, but it requires serious user-interface work. 

John Balla is a senior analyst and Jennifer Harty is an analyst at 
Doculabs, an independent advisory firm. They can be reached at info@ 
doculabs.com. Jeff Angus is senior technology editor at InformationWeek. 
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When a congressman blatantly Ignores the public interest and votes 
for legislation that benefits a major campaign contributor, we often deem 
it business as usual. When a corporate executive makes decisions based on 
the short-term value of his stock options rather than the long-term 
financial interests of the business, we shake our heads and conclude that 
it's just human nature. But when an IT manager lobbies for more resources, 
nurturing organizational alliances and making deals to garner support, we 
don*t quite know what to make of it. After all, maneuvering of this type 
could diminish the probability that IT decisions will be made rationally, 
based on a thorough assessment of costs and benefits. 

While cost/benefit analysis may make textbook sense as a principle 
guiding decision-making, it is woefully inadequate in explaining the way 
modern organizations operate. Increasingly, the IT decisions made by 
managers are influenced by non-technical factors in an environment where 
managers at all levels are struggling to effectively leverage PCs and 
other equipment that allow market leaders to make information a strategic 
asset. Information technology is the top issue on senior management's 
agenda. And most senior managers have become believers: It's impossible to 
be successful if your IT infrastructure is in ruins. 

Meanwhile, engineers and programmers frequently appear oblivious to 
the strategic issues that keep senior management awake at night. After 
all, they're busy enough trying to find the best way to push those bits 
down the wire a little faster. To borrow from popular psychology, we might 
say that senior management is from Mars while the IT organization is from 
Venus. It's time for IT managers, and network managers in particular, to 
recognize that success is not always the by-product of being right. 
Information management has become the most politicized issue of the modern 
organization, and to be successful themselves, the techies need to get in 
the game. 

Politics Isn't Always a Dirty Word Mention the word politics to a 
typical network engineer or systems administrator and you'll evoke the 
same response you'd get if you dragged your fingernails across a 
blackboard. While that view of the world is understandable, it's founded 
on a biased perspective on politics' role, broadly defined, within 
organizations. 

Senators, governors and mayors are paid to make decisions about 
allocating resources-dividing up the public pie, so to speak. Sometimes, 
issues lend themselves well to decisions based on thoughtful analysis, and 
this fosters rational decisions that any well -informed person could 
support. More often than not, however, decision -makers base their choices 
on incomplete information and the desire to serve multiple interests. 
Occasionally, they serve their own interests-a strategy most of us would 
regard as lacking in principle. Other times, it's not crystal clear which 
interests are being served because the cause-and-effect relationship 
between decision and outcome is complex. 

The allocation of resources within an organization is its politics. 
At any given moment, we can take a snapshot of how resources are 
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distributed and identify a power structure that both drives, and benefits 
from, that distribution. But the power structure in organizations is often 
fluid, influenced both by market variables and internal processes, 
Increasingly, information is an asset that threatens the status quo power 
structure. Information can transform the organization, leaving in its wake 
a group of once-powerful managers who resisted adoption of modern 
technologies. 

IT managers are finding themselves in the midst of this tumult. If 
passive, they are seldom well-regarded either within or outside the IT 
organization. That's because of a powerful, if not altogether rational, 
belief that you can't get ahead making conservative IT decisions. But if 
IT managers are too aggressive in advocating a particular direction, they 
risk disaster if things don't pan out. IT managers, like casino gamblers, 
often need a little luck. 

The Assertive Senior Management Team With big money on the IT table 

, it should come as no surprise that the senior management team, maybe 

even the chief executive, has taken a keen Interest in all this IT 

wizardry. The middle-class knowledge elite are implementing LANs to 

interconnect their home PCs, and television commercials brainwash us into 

believing that the Internet and e-commerce will revolutionize our economy. 

The scary part is that it's probably true. The odds are high , and getting 

higher every day, that at least one or two of the managers on the 

executive team have become technology junkies, installing the latest 

Windows NT service pack and reading The Wall Street Journal while waiting 

for setup to run its course- 
Is this a good thing or a bad thing for IT managers? Well, it's 

good to the extent that you may feel less pressure to dumb-down a 

technical explanation of why some new or exotic technology isn't ready for 

prime time. The IT vocabulary that managers used to leave at the office 

when they presented to senior management is becoming a more accessible 

element of our lexicon. And it may help you get enough of a budget boost 

to keep your best people without the need to resort to Y2K fear-mongerlng. 

But It spells trouble because It Increases the likelihood of conflict over 

strategic IT directions. 

Now, you have senior management Interested in IT. You also have 
technical staff who advocate differing directions, partly because they're 
self-confident, partly because their Increasingly specialized IT skills 
can make them indispensable or redundant-depending on which road you take. 
And of course, you have all the department and division managers, often 
with their own technical experts (the ones who install their NT service 
packs for them), busily optimizing internal systems to meet their own 
distributed needs. And they aren't overly concerned that this procedure 
doesn't follow organizational standards. 

The IT Manager Merry-Go-Round It Isn't easy for IT managers to 
survive in such an ambiguous environment, particularly when technology 
companies are making a nasty habit of meeting delivery targets with shoddy 
hardware and software, or worse yet, not meeting them at all. Even If a 
tenuous organizational consensus can be assembled, will the underlying 
technology work as advertised? Based on my experience, I'd say It probably 
won't, at least not within the time frame originally envisioned. 

Good IT managers burn out quickly In these circumstances, but with 
today's shortage of people who have both technology and management skills, 
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the good news is that the free-agent market is booming. The organizational 
politics associated with reconciling intra- or inter- team conflicts can be 
debilitating to the best of them. And when senior management intervenes to 
demonstrate leadership, it's often in the form of technology strategies 
that may be long on practical business skills but short on technical 
common sense. Forced migrations to NT are perhaps the decade's freshest 
examples. 

But so what if senior management drives out the best and brightest 
among the IT elite by setting bad direction and policy? Such a situation 
is custom-made for an outsourcing deal, usually the kind that's slanted 
toward the outsourcer, who must author a reasonable balance sheet while 
maintaining overhead much higher than that of an in-house organization. 

Is There Any Hope? Some days, I wonder if there is a way out of 
this mess. It's easy to lose hope. But then I observe the positive 
transformations that so many organizations have made through smart IT 
adoptions. These implementations may not have been efficiently managed , 
and some problems may have to be worked out, but ITs forward momentum is 
apparent for several reasons. 

First, organizations able to find and retain good management have 
become more efficient or have dramatically enhanced their market position 
through the strategic adoption of technology. More often than not, this is 
the result of a visionary who refuses to succumb to failure. Good economic 
times have enabled many organizations to pay high salaries, often above 
market, and money is a talent magnet. The trend in this regard grows as 
the financial allure of a career in IT has begun counteracting the 
perception that it's a field for geeks with pocket protectors. 

Second, the legacy operation is often so abysmal that even the most 
basic technology upgrades can have a significant favorable impact. Imagine 
a return to typewritten manuscripts, clackety old cash registers, ATM-less 
shopping plazas, and no Internet. In this respect, the investments in IT 
have clearly paid off, even when they are not necessarily leading-edge. 

Perhaps most important, there is evidence that the IT field is 
maturing, and that traditional battle lines are eroding. Mainframe and 
client/server experts are now meeting and engaging in constructive 
dialogue, each one fairly knowledgeable of the other's responsibilities 
and the appeal of their favored systems. Battles about the best network 
protocol or processor design are now assuming a more pragmatic bent. 
Relinquishing past hostilities, many experienced IT staff increasingly 
welcome the views of consultants, recognizing that the view of an outside 
expert can often lead to common ground, albeit at a fairly high price. And 
as senior management gains a greater comfort level in dealing with 
information-oriented issues, a more realistic view of the costs and 
benefits will emerge. 

Send your comments on this column to Dave Molta at dmolta@nwc.com. 
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Every software vendor tries to sell high up in customer 
organizations, but Comshare takes this concept to extremes. The president 
and founder of the leading executive information systems ( EIS) vendor 
spends much of his day selling systems to top executives. 

"At first we were amazed when we learned there were seven executives 
on a jet coming out to Ann Arbor," says Richard Crandall, president of 
Comshare and a fixture in the computer field for more than two decades. 
The amazement soon subsided as Learjet after Learjet landed at the Ann 
Arbor, Mich., municipal airport to view demonstrations of Commander EIS, 
the firm's mainframe and PC system introduced in 1 985 for bringing 
executives into the electronic age. 

The steady flow of visitors has helped propel Comshare, founded by 
46-year-old Crandall and several other University of Michigan graduates in 
1 966 as a timesharing service center, to the top of the $30 million to $40 
million EIS business with 49.3% of the market in 1988, according to a 
recent study by the Gartner Group in Stamford, Conn. Although Comshare 
earned most of its $88 million in revenue for the fiscal year ended in 
June 1989 from its timeshare and decision support businesses, Framingham, 
Mass.-based International Data Corp. expects Comshare to profit from an 
explosive growth in the EIS market, which should top $200 million in total 
revenue in two years. 

Not that the company is without competition. Pilot Executive 
Software of Boston is considered a strong contender, with 26.7% of the 
market, according to the Gartner Group. And relative newcomers Execucom 
Systems Corp. of Austin, Texas, and Information Resources Inc. of Waltham, 
Mass., are making concerted efforts to corner the market. Even IBM is 
jumping into the act this month with its first EIS product, Executive 
Decisions. 

Overcoming MIS Fears 

"With EIS, early on it became very necessary to be one on one with a 
sponsoring executive because half the conversation has nothing to do with 
the product," Crandall says as he reads Commander-generated memos from a 
large touch-screen monitor encased in an oak cabinet, the only high-tech 
fixture in his spacious office. 'They ask, 'Will this fit with our 
corporate culture?' and need to overcome fears and biases." 

In addition to courting CEOs, Comshare is pursuing the support of IS 
management. The company initially sold exclusively to senior management 
but realized that IS acceptance is critical to EIS's success. 

Crandall admits that selling the IS director on EIS has not always 
been as easy as convincing the CEO. "In approximately half the cases, the 
key IS chief or CIO is somewhere in the path of moving toward strategic 
systems or of blending technology with a corporate strategic plan," 
allowing ambitious projects to get off the ground quickly. But , he adds, 
"there is the other half-and it was fully half the last time I looked-that 
believe their executives wouldn't go for it or feel that this is Just one 
more Trojan horse for uncontrolled end-user computing. In some cases it 
goes in anyway, and then these become difficult customer relationships to 
manage. 

'The concern is that IS will pick up a lot of additional 
responsibilities in reporting that it didn't have before, in timeframes 
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that are very unrealistic," admits Crandall. "We are not just talking 
about end-user business professionals that may have demands but no power. 
We are talking about people who have demands, timeframes, and lots of 
power. That's a frightening situation if you are not convinced that you 
have the toolset that allows you to deliver the applications in time." 

To assuage those concerns about the system, which sells for $300,000 
to $500,000, Comshare has spent the past four years enhancing Commander's 
toolset. Among the latest development efforts is a Macintosh version due 
in the fourth quarter of this year, a Presentation Manager version 
expected around year end, and a joint project currently under way with Dow 
Jones & Co. to provide easy access to public access databases. 

Gary Gulden, senior VP of Cambridge, Mass.-based Index Group, says 
of Comshare: 'They are very heads-up players, and they look outside their 
firm for opportunities. They have a large R&D commitment, and that 
commitment finds Its expression not just in their own products but also In 
connections with other products." 

Comshare, In fact, recently laid out a five-year plan in which It 
will support Digital Equipment Corp.'s VAX environment and Apple products. 
"Our belief is that customers are increasingly looking for choice in their 
strategic selections of hardware, software, and telecommunications," says 
Crandall, 'They want to minimize the vendors they deal with. They want 
flexibility, and they want to avoid being locked in to super proprietary 
applications. . .We have modified our strategic plan and have been heavily 
influenced by customer demand to support a broader set of platforms than 
IBM." 

Crandall emphasizes, "It Is in Comshare's strategic plan to be 
complimentary to the IBM environment," and Commander Is compliant with 
OS/2 and SAA. But as for other platforms, he says: "We had a lot of 
management divisiveness about who we should support. There was strong 
advocacy for Unix and Apple, 

"We've been very impressed with the genuine incursion of Apple into 
the IBM customer set. We've had customers like E.I.! Du Pont de Nemours & 
Co.!, which are very key IBM accounts, press us to the wall to support 
Macintosh for Commander. I've never received the kind of correspondence 
from as high a level-CFOs and ClOs-on any subject than I have on Commander 
supporting Apple." 

The Pilgrimage To Ann Arbor 

The Du Pont executives made the pilgrimage to Ann Arbor in 1 988 and 
generally liked what they saw. "It was extremely refreshing, because these 
were people who understood the market we were dealing with and not just 
the technology," says Tom Holmes, product manager of EIS and a member of 
Du Pont's EIS steering committee, 'They help you understand your internal 
marketplace, as it were, which are the executives," 

Like most managers who develop and oversee EIS, Holmes is a hybrid 
IS/business manager. Though he has many years of marketing and business 
experience, he reports to Du Pont's VP of information technologies, 
Raymond Cairns. Du Pont executives have used decision support systems for 
more than six years, but when then-vice chairman Edgar Woolard Jr. (now 
chairman and CEO) asked to explore EIS, Cairns says, a steering committee 
of members from several departments was organized. 

Cairns says a major requirement was for the system to tap Into the 
existing flow of data such as Profs messages on the corporation's IBM 
mainframe and AII-in-1 documents on its VAXes. "I can get directly into 
AII-in-1 from Commander," he says. 

"We tried to find a package that required an absolute minimum of 
work," he adds. With only a half dozen people working on the system, " we 
feel we have met that goal." Furthermore, most of this staff is 
continually enhancing the system. 'The system has to continually evolve," 
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Cairns says. "It is never done, and that is what's so exciting about it." 

The chemical and energy conglomerate currently disseminates data on 
market trends, competitive analysis, human resource activity, and 
financial results, which more than 100 executives in the United States , 
Switzerland, and the Far East can download, analyze, and redistribute. 
Says EIS manager Holmes, "We expect to expand our current applications to 
a couple of hundred executives and penetrate Latin America, as well as 
Canada, the rest of Europe, and more of the Far East." 

Currently, 1 0% of Comshare's business comes from the Pacific Rim, 
where developmental efforts are under way to support Fujitsu, and the 
balance is about equally divided between Europe and North America. 
Crandall notes that 30% of Comshare's sales are in Europe, unusually large 
for an American company. 

But before It can expand, Comshare's first order of business is to 
add more of the same type of marketing information now available, explains 
Holmes. 'We probably need to Increase the ability to dissect the 
information from a regional basis as well as a global basis," he says. 
"That will take work on our part to get the data correct." 

A familiar refrain from companies such as ElS-savvy Du Pont is that 
the system is quickly spreading down into lower levels of management and 
into individual business groups. "We keep getting surprised at the 
applicability of these tools lower in the organization," he says. "At 
first we thought only 1 0 people at Du Pont would want the information , but 
we quickly saw this assumption fade away. Just because you are lower in 
the organization doesn't mean you don't want and need easy iconic access 
to information." 

Crandall does not take much credit for the spread of EIS beyond the 
board room. "It's not something you have to tell anybody to make happen," 
he says. "It happens all by itself. When a manager is getting some 
information, the people who report to him want to get that information-and 
in more detail and sooner." 

Du Pont's Holmes says he values Comshare's policy of listening 
carefully to customer requests. "We like to establish good partnership 
relationships with our key vendors," he says. "We like to influence their 
development. They in turn get to do more business with Du Pont." 

IS director Cairns says EIS changes some-but not all-the ways Du 
Pont does business. "When trying to be a truly global company, it's very 
important that information is available simultaneously in the U.S., the 
Far East, and South America and Europe," he says. "By being consistent, it 
has influenced the decision-making process." At the same time, "we wanted 
to present information that was consistent with how the company operates," 
he says. "We didn't want to give people access to information prior to 
their subordinates. That way it becomes a non-threatening system." 

High Risk, High Reward 

The efforts of Comshare and its competitors to improve their 
products seem to be slowly winning over IS managers, say observers. " In 
the past, it was senior executives who were the strong advocates for EIS. 
And while that is still very important in many organizations, MIS is now 
taking a proactive stand in terms of encouraging management to consider 
EIS," says Hugh Watson, director of MIS programs at the University of 
Georgia in Athens. "I think EIS will be increasingly important and common, 
A failure of MIS to get involved in it now will be a greatly missed 
opportunity." 

But at the same time, EIS has to date been a fairly risky 
proposition, Watson warns. A study he conducted this year for the 
University of Georgia's Executive 2000 center for research into executive 
activity contacted 50 firms with EIS projects and found 20 that had 
failed. Watson says his staff is now taking a closer look at why these 
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failures occurred. 

Despite the risks, EIS is an easy way for IS managers to demand 
attention. "EIS offers one of two approaches currently available for MIS 
to have high visibility in the firms," Watson adds. 'The other one , of 
course, is to use information technology for competitive advantage." 

Jack Rockart, director of the Center for Information Systems 
Research at the Massachusetts Institute of Technology's Sloan School of 
Management and co-author of Executive Support Systems, says the broad 
experience of those being put in charge of large EIS systems is a positive 
sign, "In the most significant systems we see, the person who tends to be 
running these things as the operational sponsor is someone with different 
types of knowledge of the company," he says. And, of course, "they are 
selected for their ability to communicate with senior executives." 

In addition, he says, finance and administrative personnel are 
coming to better terms with IS. 'There is a clear emphasis on working with 
the data suppliers," he says. 

Rockart also sees a subtle shift in the profile of today's CEO or 
CFO that has heightened the importance of EIS. 'There is a rising tide of 
senior executives who realize they cannot be hands-off," because of 
tougher global competition and shrinking middle management, he says. "A 
number of companies are worried about coordinating their strategic 
business units. I believe we are seeing a number of senior executives who 
are not financial experts but operational experts and who really want to 
be on top of the business." EIS is one method of reaching that goal. 

Tapping The Right Resources 

While Comshare is branching out into new platforms such as the 
Macintosh, many observers credit its success with early and thorough 
support of true Blue architectures VM and MVS. "What Comshare has done is 
to target the information to the IBM environment, which is where most 
corporate data is stored," says Larry Runge, program manager with GE 
Capital's Fleet Services, who previously managed EIS environments at GE 
Capital's aeronautics business. At GE Capital, strategic mainframe 
information is generally stored in applications such as Profs, DB2, and 
Comshare*s System W decision support package. 

Fleet Services' general manager is expected to use a Commander- based 
system currently in prototype, to track more than 400,000 leased trucks 
and vehicles in order to gather and analyze financial and marketing 
information. Runge's system presents frequently updated spreadsheets of 
Fleet data, with problem areas automatically highlighted in red, causes 
for concern in yellow, and strong performance figures in green. The system 
reduces the time it takes for a report to reach an executive and for that 
executive to analyze and respond to it. 

"Exception reporting really isn't practical without some automated 
tool because you end up sifting through reams of data to identify 
problems," he says. And whereas, "in the past, middle management would 
prepare reports for delivery to upper management, which takes time, now 
the executive has access to the latest numbers just as soon as they are 
entered into the mainframe," 

One gap in the Comshare product line is the lack of a Unix version 
of Commander, But Mary Danforth, VP of product development, says requests 
for it have thus far come only from a handful of government and European 
clients, 'There are problems with Unix being the business base," she says, 
citing security. Crandall says Comshare is not supporting Unix because, in 
his view, "the numbers aren't there." Still, Danforth says Unix is one of 
the next ports under consideration at Comshare. "We see the AS/400 and 
Unix as being where we may move to," she says. 

In addition to diligently supporting IBM architectures, Comshare 
struck an agreement with IBM in 1 988 for the two sales forces to combine 
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efforts on certain accounts. Crandall says this has been beneficial but 
nonetheless involved a choice between short-term profit margins and 
long-term market penetration. "Sometimes we have a little difficulty from 
an economic standpoint because our sales force will work as hard on an IBM 
deal as they will work without IBM's help," he says, "and the theory is 
that there should be greater productivity. Still, the opportunities to 
reach more clients through the partnership offsets this." 

Comshare also struck up a value-added reseller relationship this 
January with Atlanta-based Management Science America, which sells a 
$50,000 entry-level version of Commander that runs with MSA's own 
Brightview PC-based graphical front-end for mainframes. Norman Neuman Jr., 
Comshare's VP of U.S. marketing, says MSA's broad applications support 
experience allows the product to tackle more traditional operational 
projects tailored to specific industries. "What they give us as a VAR is 
the client relationships and more sales people," he says. 
Instant Response 

Hertz Corp. is one Comshare client that has benefited from EIS's 
speed of response, an essential tool in the highly competitive and 
variable rental car business. 'This is the example of a competitive market 
in the capitalist system if there ever was one," says William Carroll, VP 
of marketing planning for Hertz. He adds that EIS "allows our senior 
managers to integrate a great many complicated decisions systematically," 
such as location and pricing of cars. "It allows you to understand much 
better the impact of market segmentation on your business," he says. 

Traditional reporting systems can assemble transactions into a few 
standard aggregates for a crude look at overall rental activity, but EIS 
is far better at probing business within regions and at performing 
"what-if" scenarios based on many different elements, such as seasonal 
variation, the mix of different types of business, and economic factors. 
"The users are capable of re-aggregating the information in a variety of 
ways that allow them to come to conclusions, develop strategies, and 
exploit opportunities," he says. Exactly what executives learn about the 
market "is something we prefer not to describe in detail for competitive 
reasons," Carroll says. 

To keep overhead to a minimum, commonly used reports are created and 
stored at the mainframe, ready for distribution. For processing reports on 
a specific subject. Hertz executives simply request chunks of raw data 
from the mainframe for local charting and "what-if" analysis in 
Commander's built-in spreadsheet or in a third-party application such as a 
statistical analysis package. 
The Fuzzy Economics Of EIS 

While at Hertz the sheer necessity of staying up to date on 
marketing information points up the worth of the company's EIS, 
calculating the cost effectiveness of such a system with precision can be 
elusive. Says GE Capital's Runge: "It's hard to quantify how EIS is 
contributing to executive decisions. It's probably just as difficult to 
quantify how executives arrive at decisions." 

Lynda Applegate, an assistant professor of business administration 
for Harvard Business School who has studied and advised companies about 
EIS, cites as an exception one company that was able to cut inventory as a 
result of having more timely information through EIS. The capital freed up 
by the reduced inventory was easily quantified and came to more than the 
EIS investment. 

In most cases, managers must look for EIS's broad effect on the way 
their company does business. Adds Applegate, "You see the dollar benefits 
when you can change your business strategy." 

That may seem like a subjective reason for plunking down nearly half 
a million dollars for an EIS system, but Comshare watchers say it should 
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be enough to keep the company racing forward. Says Bill McNee, analyst at 
the Gartner Group: "I am very high on the company. They are well 
positioned for the future." 

Hot Competition For Control Of EIS Market 

Comshare may be doing well in the EIS market, observers agree, but 
its hold on the top spot is hotly contested by the other major presence in 
the field, Pilot Executive Software of Boston. 

Pilot, which makes Command Center EIS, stepped up the rivalry 
recently with an advertisement that states: "Real leadership is not won by 
words or claims. It's built on references. Real corporations using real 
systems. Ask for our complete user list. Then ask Comshare for their list 
of M 0,000 EIS users,' " 

International Data Corp, of Framingham, Mass., estimated Comshare's 
1 987 share of the EIS software market at 52%, while it pegged Riot's at 
36%. Meanwhile, Boston-based Gartner Group's market share figures show 
Comshare climbing from 46% to 49% between 1 987 and 1 988 and Pilot edging 
down from 32% in 1987 to 26% in 1988. David Friend, who founded Pilot in 
1 983 and serves as president, disputes these figures, saying they are 
poorly substantiated. 

No one disputes that Pilot is fighting back with product 
improvements, though. This summer the company delivered an MVS version of 
Command Center, which is considered critical for breaking into the larger 
Big Blue shops, and last month it added Executive Mail, an E- Mail package 
based on easy-to-use icons and menus with interfaces to Profs and 
All-in-1 . 

Pilot also announced the fourth-quarter release of IMpact, a 
software package for issues management, or the handling of high- priority 
projects. Finally, the company demonstrated a Macintosh version of Command 
Center's front end, due out in the second quarter of next year. Other 
projects in Pilot's ambitious development schedule for 1 990 include a 
back-end version for Unix hosts and a front-end one for OS/2 with 
Presentation Manager. 

Clare Gillan, senior analyst at IDC, stands by her company's study 
but notes that claims of revenue leadership by one company and user base 
leadership by another need not be contradictory. Comshare's policy of 
charging a company for each workstation user, she says, seems to have 
contributed to its bottom line, while Pilot's policy of granting 
comprehensive site licenses appears to have encouraged corporations to 
spread that company's software throughout the corporation. 

Observers agree that both firms are far ahead of all others, not 
only in market share but also in the variety of functionality and breadth 
of platforms supported. Says Gillan, "Both have been In the market for a 
number of years, and both contributed to educating the market, and both 
are investing a lot of development effort." 

?ds 



Set Items Description 

51 1 125517 BAYES? OR PROBABIL? OR STOCHASTIC? OR BELIEF 

52 29316 SI (5N) NET? 

53 221 751 (EXPERT OR KNOWLEDGE) (5N) (BASE OR SYSTEM?) 

54 1 334349 SI OR S3 

55 8880 S4ANDFINANC? 

56 1 942 S5 AND (RISK OR DERIVATIVE?) 

57 1 563 RD (unique items) 

58 5088 (BAYESIAN OR BELIEF) (3N) NETWORK? 

59 6 S7ANDS8 

S10 46 S8ANDFINANC? 
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S1 1 6 SI 0 AND (RISK OR DERIVATIVE?) 

SI 2 157291 7 ANDCAUS? 

SI 3 157291 7 ANDCAUS? 

SI 4 97 S7ANDCAUS? 

SI 5 0 Sn N0TS9 



?b finbus 

29feb00 23:34:1 9 Userl 54250 Session D5.2 
$8.98 1 .497 DialUnits File2 
$0.00 12 Type(s) in Format 6 
$6.30 3 Type(s) in Format 7 
$6.30 1 5 Types 
$1 5.28 Estimated cost File2 

$2.00 0.340 DialUnits File6 

$0.00 6 Type(s) in Format 6 
$0.00 6 Types 
$2.00 Estimated cost Flle6 

$4.82 0.753 DialUnits File8 
$0.00 5 Type(s) in Format 6 
$8.80 4 Type(s) in Format 7 
$8.80 9 Types 
$1 3.62 Estimated cost File8 

$1 1 .38 0.872 DialUnits File34 
$0.00 19 Type(s) in Format 6 
$22.50 6 Type(s) in Format 7 
$22.50 25 Types 
$33.88 Estimated cost File34 

$0.21 0.079 DialUnits File64 
$0.21 Estimated cost File64 

$0.43 0.116 DialUnits File65 
$0.43 Estimated cost File65 

$0.22 0.070 DialUnits File92 
$0.22 Estimated cost File92 

$1.35 0.386 DialUnits File94 
$0.00 4 Type(s) in Format 6 
$1 .25 1 Type(s) in Format 7 
$1.25 5 Types 
$2.60 Estimated cost File94 

$0.40 0.167 DialUnits Flle99 
$0.40 Estimated cost File99 

$4.59 0.899 DialUnits Filel 03 
$0.00 1 3 Type(s) in Format 6 
$0.00 1 3 Types 
$4.59 Estimated cost Filel 03 

$1 .49 0.327 DialUnits Filel 08 
$1.49 Estimated cost Filel 08 

$5.44 1 .554 DialUnits Filel 44 
$0.00 5 Type(s) in Format 6 
$0.00 5 Types 
$5.44 Estimated cost Filel 44 

$2.1 0 0.526 DialUnits File239 
$0.00 7 Type(s) in Format 6 
$6.00 4 Type(s) in Format 7 
$6.00 1 1 Types 
$8.10 Estimated cost File239 

$0.30 0.075 DialUnits File241 
$0.00 3 Type(s) in Format 6 
$0.00 3 Types 
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$0.30 Estimated cost File241 

$3.00 0.230 DialUnits File434 
$3.00 Estimated cost File434 

$4.22 0.81 9 DialUnits File647 
$0.00 30 Type(s) in Format 6 
$8.10 3 Type(s) in Format 7 

$8.10 33 Types 
$1 2.32 Estimated cost File647 

OneSearch, 16 files, 8.710 DialUnits FileOS 
$10.00 TYMNET 

$1 1 3.88 Estimated cost this search 

$1 1 4.28 Estimated total session cost 8.869 DialUnits 

SYSTEM:OS - DIALOG OneSearch 
File 9:Business & Industry(R) Jul/1 994-2000/Mar 01 

(c) 2000 Resp. DB Svcs. 
File 1 5:ABI/INF0RM(R) 1 971-2000/Feb 29 

(c) 2000 Bell & Howell 
File 1 6:Gale Group PROMT(R) 1 990-2000/Feb 29 

(c) 2000 The Gale Group 
File 1 8:Gale Group F&S index(R) 1 988-2000/Feb 29 

(c) 2000 The Gale Group 
File 20:World Reporter 1 997-2000/Feb 29 

(c) 2000 The Dialog Corporation pic 
File 1 48:Gale Group Trade & Industry DB 1 976-2000/Feb 29 

(c)2000 The Gale Group 
File 160:Gale Group PROMT(R) 1972-1989 

(c) 1 999 The Gale Group 
File 1 69:lnsurance Periodicals 1 984-1 999/Nov 15 
(c) 1 999 NILS Publishing Co. 
*File 1 69: This file is closed (no longer updating). 
File 267:Finance & Banking Newsletters 2000/Feb 28 

(c) 2000 The Dialog Corp. 
File 268:Banking Information Source 1981-1 999/Nov W2 
(c) 1 999 Bell & Howell 
*File 268: The database will not be updated until March, 2000 
due to changes and enhancements by Bell & Howell. 
File 473:Financial Times Abstracts 1 998-2000/Feb 29 

(c) 2000 The New York Times 
File 475:Wall Street Journal Abs 1 973-2000/Feb 29 

(c) 2000 The New York Times 
File 481 :DELPHES EUR BUS 80-1 999/DEC W3 

(c) 1 999 ACFCI & CHAMBRE COMM IND PARIS 
*File 481: File 481 will not be updating In January or February. 
Please see HELP NEWS481 for more information. 
File 485:Accounting and Tax Database 1 971-1,999/Nov W2 
(c) 1 999 Bell & Howell 
*File 485: The database will not be updated until April, 2000 
due to changes and enhancements by Bell & Howell. 
File 583:Gale Group Globalbase(TM) 1 986-2000/Feb 29 

(c) 2000 The Gale Group 
File 609:Bridge World Markets News 1 989-1 999/Dec 31 
(c) 1999 Bridge 
*File 609: File 609 will not be updated until further notice 
due to some upcoming file changes and enhancements. 
File 621:Gale Group New Prod.Annou.(R) 1 985-2000/Feb 29 

(c) 2000 The Gale Group 
File 623:Business Week 1 985-2000/Feb W3 
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(c) 2000 The McGraw-Hill Companies Inc 
File 624:McGraw-Hill Publications 1 985-2000/Feb 29 

(c) 2000 McGraw-Hill Co, Inc 
File 625:American Banker Publications 1 981-2000/Feb 29 

(c) 2000 American Banker 
File 626:Bond Buyer Full Text 1 981-2000/Feb 29 

(c) 2000 Bond Buyer 
File 635:Business Dateline(R) 1 985-1 999/Nov 17 

(c) 1 999 Bell & Howell 
*File 635: Updating is expected soon beginning with Nov. 1 999 updates 
and proceeding forward. The file should be current within 7-10 days. 
File 636:Gale Group Newsletter DB(TM) 1 987-2000/Feb 29 

(c) 2000 The Gale Group 
File 790:Tax Notes Today 1 986-2000/Feb 25 

(c) 2000 Tax Analysts 
File 791 iState Tax Today 1 991 -2000/Feb 25 

(c) 2000 Tax Analysts 
File 792:Worldwide Tax Daily 1 987-2000/Feb 25 

(c) 2000 Tax Analysts 

Set Items Description 



?s (bayesian or belief) (5N) network? 
Processing 

Processed 10 of 26 files ... 
Completed processing all files 
3871 BAYESIAN 

261578 BELIEF 

4441236 NETWORK? 

51 1 1 87 (BAYESIAN OR BELIEF) (5N) NETWORK? 
?s si and risk and derivative? 

1187 SI 
1864583 RISK 
229427 DERIVATIVE? 

52 2 SI AND RISK AND DERIVATIVE? 
?t S2/6/1-2 

2/6/1 (Item 1 from file: 15) 
00915729 95-65121 

**USE FORMAT 9 FOR FULL TEXT** 
Social information processing and social networks: A test of social 
influence mechanisms 
Sep 1994 LENGTH: 35 Pages 
WORD COUNT: 13564 



2/6/2 (Item 2 from file: 1 5) 
00835943 94-85335 

**USE FORMAT 9 FOR FULL TEXT** 
Economic institutions and the satisfaction of human needs 
Mar 1 994 LENGTH: 42 Pages 
WORD COUNT: 13991 
?s s2 and risk and caus? 
2 S2 
1864583 RISK 
2670642 CAUS? 
S3 2 S2 AND RISK AND CAUS? 
?s s3 not s2 
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2 S3 
2 S2 

54 0 S3 NOT S2 

?ds 

Set Items Description 

51 n 87 (BAYESIAN OR BELIEF) (5N) NETWORK? 

52 2 SI AND RISK AND DERIVATIVE? 

53 2 S2 AND RISK AND CAUS? 

54 0 S3NOTS2 
?s si and risk 

1187 SI 
1864583 RISK 

55 203 SI AND RISK 

?s s5 and ((errors or fault? or Integrity) (5N) data) 

Processed 10 of 26 flies ... 

Processing 

Completed processing all flies 
203 S5 
235784 ERRORS 
282842 FAULT? 
247715 INTEGRITY 
6312183 DATA 
51 389 ((ERRORS OR FAULT?) OR iNTEGRITY)(5N)DATA 

56 2 S5 AND ((ERRORS OR FAULT? OR INTEGRITY) (5N) DATA) 
?s s6 not s3 

2 S6 
2 S3 

57 2 S6NOTS3 
?t S6/6/1-2 

6/6/1 (Item 1 from file: 148) 

04844980 SUPPLIER NUMBER: 089261 76 (USE FORMAT 7 OR 9 FOR FULL TEXT) 
Dangerous liaisons, (plant-wide information processing) 
August, 1990 

WORD COUNT: 3020 LINE COUNT: 00253 



6/6/2 (Item 1 from file: 609) 
2878006 13433 

QUAKE HAZARD FOR PACIFIC NORTHWEST GREATER THAN PREVIOUSLY BELIEVED 

DATE: January Fe 0 , 1 995 

?s s5 and financ? 

Processing 

Processing 

Processed 1 0 of 26 files ... 
Processing 

Processed 20 of 26 files ... 
Completed processing all files 
203 S5 
11224734 FINANC? 
S8 103 S5 AND FINANC? 

?rd 

»>Duplicate detection is not supported for File 481 . 
»>Duplicate detection is not supported for File 623. 
»>Duplicate detection is not supported for File 625. 
»>Duplicate detection is not supported for File 626. 
»>Duplicate detection is not supported for File 790. 
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»>Duplicate detection is not supported for File 791 . 
»>Duplicate detection is not supported for File 792. 



»>Records from unsupported files will be retained in the RD set. 
...examined 50 records (50) 

»>Record 623:376843 ignored; incomplete bibliographic data, not retained - 
in RD set 

...examined 50 records (100) 
...completed examining records 
S9 65 RD (unique Items) 
?t S9/6/1-65 

9/6/1 (Item 1 from file: 9) 

02588364 021 38828 (USE FORMAT 7 OR 9 FOR FULLTEXT) 

AMR and new business opportunities-not an oxymoron 
September 1 999 
WORD COUNT: 2418 



9/6/2 (Item 2 from file: 9) 
02019993 (USE FORMAT 7 OR 9 FOR FULLTEXT) 
Approval or Not, PrimeStar's O'Brien Forges Ahead 
December 08, 1 997 
WORD COUNT: 3772 



9/6/3 (Item 3 from file: 9) 
01 91 701 9 (USE FORMAT 7 OR 9 FOR FULLTEXT) 

the Future is Now ~ Leading-edge IT managers tap into vendor R&D efforts 
August 25, 1 997 
WORD COUNT: 3177 



9/6/4 (Item 4 from file: 9) 
01 8101 64 (USE FORMAT 7 OR 9 FOR FULLTEXT) 
The Dawn Of Legal Networks 
April 1997 
WORD COUNT: 3270 



9/6/5 (Item 5 from file: 9) 
01 309205 (USE FORMAT 7 OR 9 FOR FULLTEXT) 
CUT LOOSE 
October 1 5, 1 995 
WORD COUNT: 2188 



9/6/6 (Item 6 from file: 9) 
01 240641 (USE FORMAT 7 OR 9 FOR FULLTEXT) 
NO RUSH TO THE WEB 
July 17, 1995 
WORD COUNT: 826 



9/6/7 (Item 1 from file: 1 5) 
01963468 47012176 

**USE FORMAT 9 FOR FULL TEXT** 
Who is buying life insurance practices? 
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Dec 6, 1 999 LENGTH: 4 Pages 
WORD COUNT: 3093 



9/6/8 (Item 2 from file: 1 5) 
01906361 05-57353 

**USE FORMAT 9 FOR FULL TEXT** 
Knowledge creation and social networks in corporate entrepreneurship: The 
renewal of organizational capability 
Spring 1999 LENGTH: 21 Pages 
WORD COUNT: 10079 



9/6/9 (Item 3 from file: 1 5) 
01863212 05-14204 

**USE FORMAT 9 FOR FULL TE>CT^ 
Global e-commerce, local problems 
Jul/Aug 1 999 LENGTH: 7 Pages 
WORD COUNT: 4460 



9/6/10 (Item 4 from file: 15) 
01807201 04-58192 

**USE FORMAT 9 FOR FULL TEXT** 
Can big blue master some new steps? 
May 1999 LENGTH: 8 Pages 
WORD COUNT: 5090 



9/6/11 (Item 5 from file: 15) 
01742607 03-93597 

**USE FORMAT 9 FOR FULL TEXT^ 
ZIons* young lion 
Dec 1 998 LENGTH: 5 Pages 
WORD COUNT: 2285 



9/6/12 (Item 6 from file: 15) 
01720865 03-71855 

**USE FORMAT 9 FOR FULL TEXT** 
Levels of trust 
Oct 1 998 LENGTH: 2 Pages 
WORD COUNT: 1394 



9/6/13 (Item 7 from file: 15) 
01701577 03-52567 

**USE FORMAT 9 FOR FULL TEXT** 
Labour, gender and the economic/social divide 
1998 LENGTH: 14 Pages 
WORD COUNT: 6405 



9/6/14 (Item 8 from file: 15) 
01699732 03-50722 

**USE FORMAT 9 FOR FULL TEXT** 
Networks within networks: Service link overlap, organizational cliques, and 
network effectiveness 
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Aug 1 998 LENGTH: 1 1 Pages 
WORD COUNT: 6362 



9/6/1 5 (Item 9 from file: 1 5) 
01546927 01-97915 

**USE FORMAT 9 FOR FULL TEXT** 
Up and coming carve outs 
Nov/Dec 1 997 LENGTH: 5 Pages 
WORD COUNT: 2732 



9/6/16 (Item 10 from file: 15) 
01356480 00-07467 

**USE FORMAT 9 FOR FULL TEXT** 
Managing an interorganlzational network: A framework of the institutional 
meciianism for network control 
1 996 LENGTH: 30 Pages 
WORD COUNT: 1 2038 



9/6/1 7 (Item 1 1 from file: 1 5) 
01353375 00-04362 

**USE FORMAT 9 FOR FULL TEXT** 
Changes in the theory of interorganizationa! relations in marketing: Toward 
a network paradigm 
Winter 1 997 LENGTH: 1 6 Pages 
WORD COUNT: 1 3842 



9/6/1 8 (Item 1 2 from file: 1 5) 
01237714 98-87109 

**USE FORMAT 9 FOR FULL TEXT** 
On the meaning of networks 
Jun 1 996 LENGTH: 20 Pages 
WORD COUNT: 6317 



9/6/1 9 (Item 1 3 from file: 1 5) 
01231161 98-80556 

**USE FORMAT 9 FOR FULL TEXT** 
Mergers, networking, and vertical integration: Managed care and 
investor-owned hospitals 
Winter 1 996 LENGTH: 9 Pages 
WORD COUNT: 6263 



9/6/20 (Item 14 from file: 15) 
01070889 97-20283 

**USE FORMAT 9 FOR FULL TEXT** 
The impact of explanation facilities on user acceptance of expert systems 
advice 

Jun 1 995 LENGTH: 1 6 Pages 
WORD COUNT: 8879 



9/6/21 (Item 15 from file: 15) 
01022281 96-71674 
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**USE FORMAT 9 FOR FULL TEXT** 
The new network firm: A spherical structure built on a human investment 
philosophy 

Spring 1 995 LENGTH: 1 5 Pages 
WORD COUNT: 6181 



9/6/22 (Item 1 6 from file: 1 5) 
01012028 96-61421 

**USE FORMAT 9 FOR FULL TEXT** 
States now driving health care reform's future 
Apr 3, 1 995 LENGTH: 4 Pages 
WORD COUNT: 2148 



9/6/23 (Item 1 7 from file: 1 5) 
00877328 95-26720 

**USE FORMAT 9 FOR FULL TEXT** 
Bid the branch good-bye 
Jun 1 994 LENGTH: 2 Pages 
WORD COUNT: 648 



9/6/24 (Item 1 8 from file: 1 5) 
00869157 95-18549 

**USE FORMAT 9 FOR FULL TEXT** 
Automated dynamic audit programme tailoring: An expert system approach 

Discussion 

1993 LENGTH: 21 Pages 
WORD COUNT: 11515 



9/6/25 (Item 1 9 from file: 1 5) 
00869156 95-18548 

**USE FORMAT 9 FOR FULL TEXT** 
Aggregation of evidence in auditing: A likelihood perspective Discussion 
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ABSTRACT: Providing explanations for recommended actions is deemed one of 
the most important capabilities of expert systems (ES). There is little 
empirical evidence, however, that explanation facilities Indeed influence 
user confidence in, and acceptance of, ES-based decisions and 
recommendations. The impact of ES explanations on changes In user beliefs 
toward ES-generated conclusions was investigated. Grounded in a theoretical 
model of argument, 3 alternative types of ES explanations - trace, 
justification, and strategy - were provided in a simulated diagnostic 
expert system performing auditing tasks. Twenty practicing auditors 
evaluated the outputs of the system in a laboratory setting. The results 
indicate that explanation facilities can make ES-generated advice more 
acceptable to users and that justification is the most effective type of 
explanation to bring about changes in user attitudes toward the system. 
These findings are expected to be generalizable to application domains that 
exhibit similar characteristics to those of auditing. 

TEXT: Introduction 

Expert systems (ES) are computer programs capable of performing specialized 
tasks based on an understanding of how human experts perform the same 
tasks. Few ESs, however, are targeted at replacing their human 
counterparts; usually they are intended to function as assistants or 
advisers to professional people with different technical background and 
problem-solving experience (Berry and Hart, 1990; Feigenbaum, et al., 1988; 
Leonard-Barton and Sviokia, 1 988). To be useful and acceptable, it has been 
argued, an ES must not only perform at a level comparable to a human 
expert's, but also must be able to explain, in a form understandable to 
users, the reasoning processes it employs to solve problems and make 
recommendations (Duda and Shortliffe, 1 983; Moore and Swartout, 1 988; Teach 
and Shortliffe, 1981). 

Central to the issue of explanation are two unique characteristics of ES 
applications. First, ESs are often developed to help make relatively 
unstructured decisions, and a time lag may exist between when such 
decisions must be made and when their quality can be assessed. As a result, 
the acceptance of ES-generated advice is more likely to be determined by 
its reasonableness than by its correctness. Second, real-world decisions 

59 

WILKINSON ANDERSON INC. 



have practical—financial, legal, political, and social— consequences. If 
users are to remain responsible for the decisions made, they are unlikely 
to accept a system's recommendation if they do not understand its 
underlying reasoning processes (Hollnagel, 1987). An explanation facility 
provides the potential to make an ES more useful and acceptable by 
increasing user understanding of, and confidence in, its decisions and 
recommendations.(1 ) 

Past research in ES explanations has focused on establishing "existence 
proofs," i.e., solving the theoretical and practical problems of 
explanation generation (cf. AAAI, 1 990; AAAI, 1 992; Moore and Swartout, 

1988) . Given the potentially high costs of making these explanations 
available, as exemplified by the difficulties earlier research has 
encountered, there appears to be little guarantee that automated 
explanations will produce the positive impact on the uses of ESs as 
expected. Indeed, there have been documented cases in which users question 
the utility of having an explanation facility (Slatter, et al., 1 988) or 
ignore the explanation facilities altogether (Hart and Wyatt, 1 990). 

In this light, our paper reports an empirical investigation of the value of 
ES explanations to users. Specifically, a laboratory setting was employed 
to study the impact of alternative types of explanations on user acceptance 
of ES-generated advice. The scope of the study was limited to diagnostic 
problem-solving, a process in which a system's true states are inferred 
from observable but noisy data called symptoms. At the conceptual level, 
diagnosis represents an important class of problems across a variety of 
domains, and it was among the earliest candidates for ES applications 
(Clancey, 1985; Davis, 1993; Stefik, et al., 1982; Torasso and Console, 

1989) . 

The next section provides the conceptual basis for the study and develops 
the research hypotheses. The following section employs a detailed 
discussion of the research method employed. The balance of the paper 
presents the results and discusses the findings. 

Conceptual Basis of Research 

According to the American Heritage Dictionary, to explain is "1. to make 
plain or comprehensible; 2, to define, expound; and 3, to offer reasons for 
or a cause of, to justify." Within the context of using expert systems to 
solve problems, the term "explanation" has been used very loosely to cover 
almost any request for further information. For example, users might have a 
need for further operational instruction, more data, explication of terms, 
feedback, or justification of the reasoning methods used or advice given. 
Clearly, in each case both the form and the content of the information 
needed could be different, and grouping them all together under the same 
term, "explanation," would not be helpful. 

This study adopts a typology of ES explanations that has been the focus of 
prior research. Together, three types of ES explanations are identified 
(Chandrasekaran, et al,, 1 988; Clancey, 1 983; 1 993; Neches, et al., 1 985; 
Swartout, 1983; Wick, 1992):(2) 

1 . Trace, or Line of Reasoning, which refers to a record of the inferential 
steps taken by an ES to reach a conclusion. 

2. Justification, which is an explicit description of the causal argument 
or rationale behind each inferential step taken by the ES. 
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3. Strategy, which is a high-level goal structure that determines how the 
ES uses its domain knowledge to accomplish a task.(3) 

The next section introduces a conceptual model developed by Toulmin (1958) 
for the process of argumentation. This model is then applied to the three 
types of explanations to form the theoretical basis for our research, 

Toulmin's model of argument 

Stephen Toulmin (1958) formalizes a model of argument to reflect the 
"rational process" (p. 7) characteristic of human reasoning. Drawing an 
analogy from claims and argument made in the courts, Toulmin proposes the 
structure and procedures by which "claims-in-general" can be argued for and 
accepted. 

Toulmin makes a clear distinction between the "field-Invariant" and 
"field-dependent" aspects of argument (p. 1 5). Regardless of the context 
that surrounds the argument, its field-invariant aspect consists of six 
elements (also see Ehninger and Brockriede, 1 960; 1 978; Toulmin, et al., 
1984): 

1 . Claims (C) —assertions or conclusions put forward for general 
acceptance. A claim is always of a potentially controversial nature. 

2. Data (D)~statements specifying the particular facts or previously 
established beliefs about a situation based on which a claim is made. 

3. Warrants (W)— statements that justify (or certify) the reasonableness of 
leaping from data to a claim. 

4. Backing (B)~the general body of Information or experience that assures 
the trustworthiness of a warrant. Backing Is not needed unless the validity 
of the warrant is challenged. 

5. Qualifiers (Q)~phrases expressing the degree of certainty placed on a 
claim. No qualifier Is needed If a claim Is considered Indisputable. 

6. Possible Rebuttals (R)~extraordinary or exceptional circumstances that 
might defeat the warranted claim. The function of a rebuttal is analogous 
to a safety valve and therefore is optional. 

Argument is typically made in such a manner that the following 
relationships among the six elements exist (Ehninger and Brockriede, 1 960, 
p. 45): 

(Diagram omitted) 

Despite this basic, context-free structure said to be at work In any 
argument, according to Toulmin (1958), an analytical framework of argument 
is not complete unless it Includes the larger human enterprise whose 
purposes argument serves; hence the field-dependent aspect of argument. For 
example, in different domains, there are differences in the degree of 
formality and precision that argument must satisfy in order to be 
acceptable. The fundamental force of medical argument Is realized only to 
the extent that the enterprise of medicine Itself is understood, and the 
same is true in business, politics, law, or any other fields. 

Explanation as argumentation 
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Toulmin's model of argument is significant in the way it highlights the 
discrete response steps that an ES explanation facility should follow in 
order to answer user queries in a convincing way. Increasingly, researchers 
of ES explanation facilities have drawn upon studies in argumentative 
dialogues to generate explanation dialogues (e.g., Cavalli-Sforza and 
Moore, 1992; Moore and Swartout, 1989; Quillici, 1991). The relevance of 
the model to each type of explanation is discussed next, with examples 
drawn from the domain of auditing. 

Trace 

The opportunity of using trace as a source of explanation first appeared 
with the introduction of rule-based ES (Buchanan and Shortllffe, 1984). A 
rule is a representation of a data-conclusion association, where a 
conclusion can be inferred from a logical combination of some premises. 
Moreover, certainty factors (CF) can be added to a rule to indicate how 
strongly the premises have been confirmed or how probable the conclusion 
can be inferred from the premises. Because the conclusion clause of one 
rule can be used to confirm the premises of another rule, a collection of 
rules used together become a network of Individual inference steps. A trace 
is a record of the system's run-time, rule-invocation history. Properly 
presented, a trace allows interactive question-answering about the system's 
reasoning steps: "How a request for data is related to a goal, how one goal 
leads to another, and how a goal is achieved (Clancey, 1983, p. 217)."(4) 

A rule represented as a Premise-CF-Conclusion triad corresponds to a Data 
(D)-Qualifier (Q)-Claim (C) structure in Toulmin's model of argument. The 
other three elements of the model are missing, however: warrant, backing, 
and rebuttals. Trace, which provides a chain of rules invoked by an ES, is 
insufficient to explain the system's reasoning processes, unless for each 
rule in the chain the user already understands exactly why the premises (D) 
necessarily, or conditionally if the CF (Q) is less than 1 00 percent, lead 
to the conclusion (C). 
Justification 

By encoding large chunks of knowledge as empirical associations, ESs are 
constructed for efficient problem solving at the expense of omitting the 
intermediate reasoning steps on which these associations are based. 
Consequently, such systems lack the support knowledge necessary for 
justifying to the user why, for example, a conclusion follows naturally 
from its premises. To illustrate, consider the following excerpt from the 
verbal protocol generated by an auditor while performing an audit task: 

The cash overdraft always raises red flags. It's one of the items that we 
always look at. Or maybe I shouldn't say that it always raises a warning 
flag, but when people get into a cash overdraft situation then there should 
be current reasons for it. Otherwise you should be aware that they're 
having liquidity problems. 

Viewed as argument, the auditor's line of reasoning corresponds to the 
following Toulmin structure, 

(Structure diagram omitted) 

Because the auditor does have some reservations about the conclusion, a 
qualifier (Q) and a rebuttal (R) are included to serve as an escape clause. 
On the one hand, it would be rather straightforward to encode this chunk of 
knowledge into a rule that, if reproduced by an ES to answer questions, 
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becomes part of a trace. On the other hand, the reason that a cash 
overdraft should signal a liquidity problem in the first place is less than 
obvious. To interested observers, the problem-solver's "inferential leap" 
from the data to the claim needs to be justified. 

Probed further, the auditor may offer new information to support his/her 
claim by adding a warrant (W) and perhaps even a backing (B). The first 
diagram on the next page completes an example of Toulmin's model of 
argument at work. 

There are two practical reasons that justification, which requires a deeper 
understanding of the domain, can be important. First, by demonstrating that 
the conclusions developed by the system are based on sound reasoning, 
justification helps increase ES users' confidence in the system's 
problem-solving competence and hence, the acceptability of the conclusions. 
Second, because ESs can only achieve high performance within relatively 
narrow problem areas, justification enables users to make more informed 
decisions on whether the system's advice should be followed. For example, 
in boundary cases or unusual situations, the lack of specific knowledge may 
cause an ES to respond to a problem inappropriately, and justification 
forces it to reveal such limitations. 

(Diagram omitted) 

Both the warrant and the backing of argument serve the function of 
justification, but at different stages. If the warrant is challenged, the 
backing becomes the next line of defense, opening a new round of argument. 
The process does not end until either the warrant is accepted, or no 
further backing can be offered.(5) Toulmin's framework shows where 
justification for a line of reasoning should be focused. Examined within 
the present context, it also points out the type of information a knowledge 
engineer needs to extract from a problem solver, if the knowledge obtained 
Is to be used later for explanation purposes. 

Strategy 

instead of asking what knowledge is being applied by an ES (trace) and its 
underlying rationale (justification), users may be interested in knowing 
why information is gathered in a certain order, why one particular chunk of 
knowledge is invoked before others, and how individual reasoning steps 
contribute to a high-level objective. To answer these questions, the system 
must have access to knowledge about its problem-solving strategy. Moreover, 
because strategic knowledge often involves general principles in problem 
solving, it also finds use in knowledge-based tutoring systems that teach, 
among other things, high-level reasoning and problem-solving skills within 
specific domains (e.g., Clancey, 1 986; Nickerson, et al., 1 985; Regian and 
Shute, 1992). 

Structurally, explaining the system's strategy is similar to providing 
justification in the sense that the system must clarify why it solves a 
problem by following a specific procedure (Clancey, 1 993). The Toulmin 
diagram below provides such an example (qualifiers and rebuttals omitted). 

(Diagram omitted) 

Cost of automated explanations 



ES explanation facilities that provide users with access to justification 
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or strategic knowledge are more difficult and more costly to develop than 
those relying primarily on a trace of the system's execution paths. The raw 
materials from which a trace is constructed are part of the system's 
knowledge base, and providing a trace as explanation is largely a problem 
of presentation. Rules, for example, are often encoded internally in a 
succinct way for efficient storage and execution, and making them 
comprehensible to a user typically involves simple translation using a 
standardized natural-language template (Barr, et al,, 1989). 

On the other hand, because strategic knowledge tends to be buried 
Implicitly in the knowledge base, and an ES does not need a representation 
of justification knowledge in order to execute, neither type of information 
is readily available for explanation purposes. Not only must additional 
efforts be spent uncovering the knowledge to be used for explanation, but 
more fundamental issues may arise concerning the psychological validity of 
the explanation knowledge so acquired. 

It is common practice, for example, to study decision making by a variety 
of process tracing techniques, i.e., observing the human expert performing 
a task and then analyzing a record of the entire problem-solving process 
(Todd and Benbasat, 1 987). It is considered rather disruptive, however, to 
ask the expert for an explanation of his/her reasoning while the 
decision-making process is under way (Ericsson and Simon, 1 984; Nisbett and 
Wilson, 1977). An alternative is to ask the expert to provide a 
retrospective rationalization of the decisions made, but there is evidence 
that people are not always conscious of their belief structures and 
therefore have marked difficulties in providing post hoc justification that 
is consistent with the quality of their performance or with their 
problem-solving behavior (Berry and Broadbent, 1984; Norman, 1983; Wason 
and Evans, 1975). 

Contrary to common beliefs, a truly informative explanation facility does 
not come free with the acquisition of some associative knowledge and the 
encoding of that knowledge in a computer program for problem solving. The 
resolution of many non-trivial theoretical and practical difficulties in 
automated explanations demands a significant amount of research and 
development resources. It is important that we establish the relative 
merits and usefulness of this technology, lest we run the risk that the 
resources devoted to constructing "existence proofs" might not be 
justified. 

Development of hypotheses 

The purpose of this study is to examine the impact of ES explanations on 
users' acceptance of ES-developed conclusions and to identify the type of 
explanation that is most effective in producing such an impact. 

Unlike conventional information systems that function in relatively 
structured task environments, the nature of ES applications suggests that 
the conclusions and recommendations developed by an ES can be potentially 
controversial. To reduce the amount of uncertainty associated with ES-based 
decisions and increase the system's acceptability to users, an explanation 
facility is necessary. 

ES explanations serve the function of argument. Because the primary purpose 
of argument is to advance a problematic claim and have it accepted, for the 
argument to be considered successful, there must be a change from an 
existing belief to the adoption of a new belief on the part of the audience 
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(Achinstein, 1 983; Dretske, 1 988; Ehninger and Brockriede, 1 978; Toulmin, 
et al., 1984). The change is unlikely to take place, however, unless the 
audience agrees with the evidence (data) and endorses the principle that is 
expressed or implicit in a warrant (Ehninger and Brockriede, 1978). It 
follows that, by making available the data an ES uses to reach conclusions 
and the rationale that justifies such inferential leaps, an explanation 
facility has the potential to affect users' beliefs in the system's 
conclusions. This expectation leads to the following hypothesis: 

HI: ES explanations can affect users' beliefs in the system's conclusions. 

Of the three types of explanations discussed above, justification appears 
to have the highest potential to change user beliefs in an ES-generated 
conclusion. Strategy, while explaining why things are done in a certain 
order, provides information at the wrong level. The fact that there is a 
separate body of "domain-general" knowledge (Clancey, 1 993, p. 1 99), which 
governs the use of domain-specific knowledge, suggests that ES conclusions 
must be expressed at a level consistent with the advice the user is seeking 
(i.e., as shown in the earlier example, an ES for financial statements 
analysis will advise that the client might have liquidity problems, while 
timing the analysis of inventory market value might be an internal task of 
that system; an ES for audit planning, on the other hand, will most likely 
offer the timing of specific task performance as advice to the user). If 
strategy does not provide information at the system's conclusion level, it 
is unlikely to impact users' beliefs in these conclusions. 

Judged from the perspective of argument, if the system's conclusion is 
indeed controversial, additional information provided by trace, i.e., data 
used to reach the conclusion, may be insufficient to bring about changes in 
users' beliefs. By definition, data are pre-established facts or beliefs. 
Data alone are unlikely to provide much added value, unless (1) the user is 
unaware of the existence of such data, and (2) the user will interpret the 
data in the same way and reach the same conclusion as the system does. 

If the user does not understand the underlying rationale of an inferential 
leap that the system is making (hence the controversy), only justification 
provides the right type of information that can help bridge the gap. 
Providing justification for the system's conclusions may or may not 
completely resolve the controversy, but making it available at least 
presents the opportunity to allow the system's side of the story to be 
understood (Toulmin, et al., 1984). If users find justification most useful 
in helping them judge the acceptability of the system's conclusions, it is 
expected that they will use this type of explanation more often and will 
spend more time examining it. These expectations lead to the following 
hypothesis: 

H2: The most effective type of explanation in helping to change users' 
beliefs is justification. 

a. Users will request justification more frequently than they will other 
types of explanations. 

b. Users will spend more time examining justification than they will with 
other types of explanations. 

Research Method 

This research was conducted in a laboratory setting using a simulated ES. 
This section describes the domain, research design, development of the 
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stimulus material and the system, dependent measures, participants, and 
experimental procedures. 

Domain 

The domain of auditing was chosen for the study of ES explanations. First, 
auditing is diagnostic in nature, since the overall objective of auditing 
is to detect discrepancies between the financial statements of an 
enterprise and its true financial health. Second, audit decisions are often 
judgmental and notably consequential; the validity of many audit decisions 
is either impossible to evaluate in a short term, or is too costly to 
verify. As a result, auditors must be prepared to defend, before their 
clients and their colleagues, every conclusion they make (Ashton, et al., 
1989). 

Design 

In this study, participants evaluated the outputs of an ES performing an 
analytical review, a procedure widely employed by auditors to collect 
evidence (Blocker and Willingham, 1985). Because of the lack of access to a 
real ES for analytical review, a computer program was developed to simulate 
the user Interface of such a system. As a tool frequently employed in the 
study of human-computer Interaction (e.g., Good, et al., 1984; Gould, et 
al., 1983), simulation allows various design alternatives to be explored In 
a realistic research environment. At the same time it avoids the use of 
such technologies as natural language processing and text generation, which 
are yet to become widely available. 

A one-group pretest-post-test design was employed In this study. 
Explanation was the independent variable: the overall presence or absence 
of explanation was manipulated, and the participants' belief in the 
system's conclusions before and after they received explanation were 
measured. There are potential confounding effects associated with a 
one-group design, specifically the history effect and the maturation effect 
(Emory, 1991). In this study, however, the duration between the pretest and 
the post-test was considered too short to cause these effects. 

Material 

The stimulus material for the experiment consisted of an audit case and the 
outputs of the simulated ES (the system). Originally designed for the study 
of auditing expertise (see Johnson, et al., 1989; 1991), the case involved 
a publicly traded medical products company In which a series of frauds were 
perpetrated by senior management. Adapted for our study, the case material 
included a narrative description of the business and a set of financial 
statements and associated notes. Having been tested and improved over time, 
the case appeared to reproduce the complexity and the challenge of the 
participants' usual task environment. 

The outputs of the system included (1) a series of audit conclusions 
related to various parts of the financial statements, and (2) three 
alternative explanations for each conclusion. Two domain consultants 
assisted with the development of these outputs. First, using data collected 
from prior research, a series of conclusions were developed. Next, using 
Toulmin's model of argument, templates were developed to constrain the 
syntactic structure that each explanation type would conform to and to 
define the nature of its Information content. Three explanation texts 
(trace, justification, and strategy) were then developed for each 
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conclusion based on input from one of the consultants. To ensure a high 
quality of explanation material, the initial set of explanations so 
constructed was presented to the second consultant for critique, and a 
revised version was shown to the first consultant. This process was 
repeated until all major disagreements were resolved. 
To ensure the three explanations pertaining to the same conclusion could be 
considered equivalent in terms of their readability, the explanation 
material was further calibrated. First, the material was rated for 
subjective readability by 1 5 MBA students completing a course In auditing, 
Their ratings were used to make readability improvement to the material. 
The explanation texts were then empirically calibrated by 32 upper-division 
business students completing an auditing course. Reading rate and recall 
accuracy were employed as indirect measures of readability (Kintsch and 
Vipond, 1979). For each conclusion developed, if its explanations were 
found to have statistically significant differences in readability, both 
the conclusion and the explanations would be excluded from the material. 
The final version of the material included 14 conclusions and their 
corresponding explanations. 

System 

A typical user-ES dialogue may include the system's requests for data, the 
user's data entry as requested, the system's presentation of conclusions 
and recommendations, and the user's requests for explanations pertaining to 
any specific data request or conclusion. Because of the large quantity of 
data required by an auditor for the analytical review task, interactive 
data entries were deemed too time-consuming to be part of the experiment. 
It was assumed, therefore, that the data entry routine would be completed 
in advance and that the system would not ask the participants for 
interactive data inputs.(6) 

The resulting dialogue included the following system outputs: (a) a client 
company profile at the beginning of the review, (b) a series of audit 
conclusions related to various parts of the financial statements, (c) three 
types of explanation for each conclusion, and (d) a review summary to 
highlight the system's recommendations. Prior studies (Johnson, et al., 
1989; 1991) indicate that a typical strategy used by auditors on analytical 
review tasks is to first develop a model of the client company, and then 
identify potential audit risks associated with such a model. Because 
auditors following the strategy tend to examine client financial statements 
in their standard presentation sequence (i.e., a narrative followed by a 
balance sheet, an income statement, a statement of changes in financial 
position, etc.), the dialogue presented various conclusions in the same 
order to reflect the use of that strategy. 

To make the system look "real," intentional delays were embedded in the 
user-system dialogue, especially prior to the presentation of each 
conclusion. This would create the impression that the system needed time to 
develop the output. The amount of keyboard inputs required on the part of 
participants was minimal. In addition, an online help facility was 
available in anticipation of potential operational difficulties. Finally, 
there was a built-in routine that would capture all user keystrokes and 
elapsed time (in 1 /1 00 seconds) between these keystrokes. The system went 
through several rounds of pilot tests with academic and practicing 
auditors. The results of these tests indicated that the system would appear 
realistic to participants. 

Dependent measures 
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Change in belief was the primary dependent variable. For each conclusion 
produced by the system, participants were asked to indicate, on a 
seven-point scale, the extent to which they believed that the conclusion 
was true or reasonable, before and after they received explanations offered 
by the system. The before-explanation score measures participants* existing 
belief in the system's conclusion, and the after-explanation score measures 
their newly adopted belief. The difference between these two scores was 
used to measure whether explanations had any impact on participants' belief 
in, and hence their acceptance of, the system's conclusion (Cohen, 1989; 
Dretske, 1988). 

Two other dependent variables— choice of explanation and elapsed reading 
time-were used to measure the degree to which participants actually found 
uses for individual types of explanations. Choice of explanation was 
recorded as the frequency of requests participants made for each 
explanation type. Elapsed reading time was measured by recording elapsed 
time between keystrokes and calculating the average time participants spent 
examining individual explanations (in seconds per 100 words of text). These 
usage measures provide a more direct indicator of the relative value of 
alternative explanation types to users. 

Participants 

Twenty practicing auditors from a large public accounting firm participated 
as subjects. They ranged from staff-level accountants with two years of 
practice to audit managers with six or more years of practice in the public 
accounting profession. While it was not practical to employ randomization 
procedures in selecting the participants, participation was voluntary, and 
there was no obvious reason to believe that the sample selected was 
systematically biased in any respect. 

Procedures 

A two-hour experiment was administered individually to participants in a 
microcomputer laboratory. All instructions were provided In written form. 
The experiment was divided into two sessions: a review session followed by 
an evaluation session. During the review session, participants were asked 
to assume the role of a manager on an audit team. Their task was to conduct 
an analytical review on a set of draft financial statements of a client 
company (the audit case) and to provide a written summary identifying any 
unusual account relationships that might lead to further investigation. The 
purpose was to engage participants in active problem-solving and to allow 
them to become familiar with the case material. 

At the beginning of the evaluation session, participants were presented 
with a different context: they were asked by their firm to evaluate a 
computer-based decision aid for analytical review. Their task was to 
provide critiques on the conclusions and explanations developed by the 
system and to make a recommendation as to whether their firm should 
consider adopting such a system. Throughout the experiment, participants 
were unaware that the system they evaluated was a simulation. 

The first half of the evaluation session served a training purpose. The 
system presented eight conclusions. Participants practiced reviewing each 
conclusion and the corresponding three types of explanations. During the 
second half, the system presented six conclusions. For each conclusion 
presented by the system, participants were first asked to indicate to what 
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extent they believed that the conclusion was true or reasonable. They were 
then allowed to either proceed to the next conclusion, or freely request 
and review any of the three types of explanations. If participants 
requested at least one explanation, before they proceeded to the next 
conclusion, they were asked again to indicate to what extent they now 
believed the system's present conclusion was being true or reasonable. 
Throughout the evaluation session, participants were also invited to 
provide, in writing, free-form comments about the system's outputs. 

Results 

Change in belief 

Change in belief was the difference between participants' post-explanation 
belief and pre-explanation belief scores. Prompted with the statement, "I 
believe the system's conclusion is true or reasonable," prior to and after 
receiving explanations, participants provided ratings on identical 
seven-point scales, ranging from "1 " for "strongly disagree" to "7" for 
"strongly agree." Table 1 presents the summary statistics on these measures 
and the results of a paired T-test performed on change in belief.(AII 
tables omitted) There was a significant increase in participants' belief in 
the system's conclusions, after they reviewed explanations provided by the 
system. The result thus provided support for HI, 

Choice of explanation 

Table 2 shows the number of times a request was made for each type of 
explanation, as well as its share (in percentage) of the total number of 
requests made. The results indicate that justification was the most 
frequently requested explanation type, followed by trace. Strategy was the 
least frequently requested explanation type. Also shown in Table 2, a 
one-dimensional chi sup 2 test found significant differences among the 
total number of requests made for each type of explanation. This result 
provided support for H2.a. 

Elapsed reading time 

Table 3 presents the average elapsed time participants spent examining each 
type of explanation (adjusted for the amount of text read), as well as the 
results of a repeated-measures ANOVA test. Participants spent relatively 
little time reading strategy, substantially more time examining trace, and 
the greatest amount of time inspecting justification. The ANOVA test found 
significant differences among the three average reading times. The result 
provided support for H2,b, 

Discussion 

This section discusses the findings and the limitations of the study, and 
the implications of the results for research and practice. 

Interpretations of findings 

The results of this study are consistent with the view that explanations 
can have a positive impact on user acceptance of an expert system. After 
they had the opportunity to learn about the system's reasoning processes 
through the explanations, participants appeared more convinced about the 
soundness of the system's conclusions, as demonstrated by their increased 
belief in those conclusions. Indeed, 17 (85 percent) of the 20 
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participants' overall impression of the system was so positive that their 
final recommendations of the evaluation were all in favor of adopting the 
system. (7) 

More specifically, the study found justification to be the most effective 
type of explanation in making the system's conclusions more acceptable, as 
evidenced both by the highest frequency at which justification was 
requested, and by the unproportionately large amount of time participants 
spent analyzing it. Participants' informal comments also provided support 
for their discrete usage patterns, as a number of them suggested that they 
would always want to see the justification for a conclusion, and they 
wondered why conclusion and justification were not presented together. 
These findings are consistent with Toulmin's prediction in his model of 
argument that a potentially controversial claim cannot be supported by data 
alone, and that a warrant will be necessary in order for the claim to be 
considered acceptable (Toulmin, 1958; Toulmin, et aL, 1984). (8) 

In addition to providing support for the research hypothesis concerning the 
usefulness of justification, the results Indicate other usage patterns. 
Participants also made a relatively high number of requests for 
explanations based on trace, which provided information on the specific 
data items the system used to reach conclusions. Considering the fact that 
the system did not request user data inputs, participants' need for trace 
seems understandable. Although they were exposed to the case material 
during the review session, they might have either ignored some case data or 
failed to remember all the details. One question naturally results: would 
users still need trace if they were asked to provide all data inputs? In 
this case, users most likely would want to know why certain data Items were 
being requested, and trace would instead provide information in terms of 
the conclusions the system would try to draw based on the data supplied 
(see an earlier discussion on page 4, and footnote 4, of this paper). In 
view of Toulmin's model of argument, trace is useful because it makes 
explicit both the data used and the claim inferred from the data, but it is 
insufficient to cause changes in users' beliefs. 

Despite their role as critics of the system, participants did not appear to 
be concerned about the system's strategic knowledge. In particular, they 
spent a brief amount of time examining strategic explanation: they seemed 
disappointed with the information provided and lost interest quickly. In 
their written comments, participants suggested that one role of strategic 
explanations was to remind the auditors about the higher-level audit 
objectives to be accomplished (i.e., the "bigger picture" to be maintained) 
while analyzing a specific situation. This finding is consistent with the 
expectation discussed earlier—that information not directly supporting the 
system's conclusions will not affect users' beliefs. Beyond its role of 
providing an overall picture of the task at hand, participants also 
commented on the potential value of strategic explanations in auditor 
training. This observation again supports the view that strategic knowledge 
might be most useful in knowledge-based tutoring systems (Clancey, 1 986; 
Nickerson, et aL, 1985; Regian and Shute, 1992). 
Finally, for exploratory purposes the impact of participants' domain 
experience on their need for explanations was also examined. The collected 
data were divided into two groups: data from auditors with two to four 
years of practice (novice group) and data from auditors with six or more 
years of practice (expert group). The data were then compared. The change 
in belief scores for the two groups was comparable, indicating they were 
equally likely to be influenced by explanations. Both groups also requested 
justification most frequently, followed by trace and strategy. In that 
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order. These informal results seem to suggest that level of domain 
experience is not a factor in assessing the Impact of explanations on users 
nor in determining the most desirable explanation type. 

On the other hand, the novice group did make a significantly higher number 
of requests for explanations than the expert group. Novices made a total of 
133 requests (56 for justification, 46 for trace, and 31 for strategy), 
compared to experts* 94 requests (30 for justification, 34 for trace, and 
21 for strategy). The difference came as no surprise, however: one would 
expect that more experienced auditors should be able to understand the 
system's conclusions better and therefore have a lesser need for additional 
information. 

Limitations 

The main limitations of the study center on the scope of the experiment, 
the limited user-system dialogs, and the experimental task used. First, the 
experiment focused on a single problem domain. The generalizability of the 
results would likely increase, for example, if two or three different 
domains were studied. The resource requirements, on the other hand, would 
be substantia! in terms of developing and testing the stimulus materials 
for each domain, which would demand large amounts of domain consultants* 
time. An argument can be made, however, that ES applications in many other 
domains share the characteristics of those in auditing and therefore can 
benefit from the study's findings. For example, medical diagnosis, credit 
appraisal, insurance underwriting, and trouble-shooting complex mechanical 
or physical systems all involve the use of judgments. Decision making in 
these domains can be highly consequential, and the correctness or validity 
of these decisions is either impossible or too expensive to quickly verify 
(Torasso and Console, 1 989). 

Second, the user-system interface used in the experiment did not allow 
different problem context for explanation to be explored. For example, the 
lack of data entry requirements would not prompt potential user questions 
on why the system needed specific data (trace would have been used to 
answer such questions), and it was not possible for participants to 
volunteer information. While the user-system dialog of the study was 
designed to emphasize ease of operation and controlled data collection, the 
interpretation of the results must take these conscious design choices into 
consideration. 

Third, in this experiment the participants' main task was to evaluate a 
decision aid, not to use it to aid in decision making. While studying such 
"first-time" user behavior is Important because it helps assess the 
acceptability of an ES as judged from its explanation capability, it is 
nevertheless artificial. Caution must be exercised in deciding whether the 
findings could be generalized to experienced users, or to the real-world 
task environment in which users will truly be held responsible for any 
decisions they make or adopt. 

Implications for future research 

The efficacy of ES explanations is often intuitively assumed by the 
research community in explanation generation technologies, but not formally 
tested (except for a survey of potential ES users by Teach and Shortliffe, 
1981). By providing empirical support for the claim that explanations can 
influence users* acceptance of ES outputs, the results of this study 
enhance the legitimacy of research in explanation development technologies. 
The results also suggest that better and more in-depth understanding of the 
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process of ES explanations is warranted. In addition, further investigation 
should be conducted on the domain characteristics that make ES explanations 
necessary and helpful. In our research, the nature of the auditing domain 
might explain a strong need for explanation facilities. Future research 
should focus on a more precise identification and definition of the factors 
in an ES task environment that dictate the usefulness of an explanation 
facility. For example, such a facility may be superfluous to users if the 
quality of the system's advice can be tested immediately, as might be the 
case in ESs developed for software debugging. After all, there appear to be 
many ESs in use now that offer at best very primitive explanation 
facilities. It is also possible, however, that ESs have not been more 
widely deployed in some domains because they are considered unacceptable 
without truly useful explanation capabilities. 

This study found justification to be the most preferred explanation type. 
As a result, a different set of research questions relating to the 
effectiveness of justification can be introduced. Specifically, more 
in-depth inquiries on justification may benefit from earlier research in 
warrant, the key construct in Toulmin's model of argument. Rhetoricians, 
for instance, have studied warrants that are based on a rich variety of 
reasoning strategies, such as causal reasoning, classification, 
generalization (case-based reasoning), analogical reasoning, and the use of 
a combination of these strategies for different types of argument (e.g., 
Ehninger and Brockriede, 1960; 1978). Given the importance of justification 
to ES users, one future research direction may be to assess the impact of 
different kinds of justification and to identify the appropriate conditions 
under which each should be employed. For example, while justification based 
on causal reasoning—more likely referred to as scientific explanations 
(Achinstein, 1983; Mackie, 1980; Nettler, 1970)~might be most convincing, 
it will not always be available. In many domains, such as in stock market 
investment, the court of law, or career counseling, reasoning based on 
statistical evidence, past cases, or analogies may be a human 
problem-solver's best explanation strategy. 

This study is an initial step toward formally examining the impact of ES 
explanation on users, A potentially more fruitful but also more challenging 
direction of research will be to study the impact of ES explanations in 
real work environments. More fundamental questions should be asked about 
the utility patterns of ES explanations in such settings, and about the 
impact of explanation facilities on users' decision-making behavior and 
performance. Obviously, these questions are of greater significance to ES 
developers and user organizations—questions that cannot be answered 
readily until they are studied in field settings and over an extended 
period of time. 
Implications for practice 

The practical problem addressed in this study was the following: can ES 
explanation facilities influence users' decisions on whether to adopt the 
advice of the system, and what kind of explanation is most effective in 
influencing users' decisions? From an ES developer's perspective, the 
study's findings suggest that explanations have the potential to make the 
system's conclusions more acceptable, if the underlying applications 
demonstrate characteristics similar to those of the auditing domain (see 
earlier argument under the Limitations section). 

The results also suggest that developers of explanation facilities for an 
auditing ES might consider making justification available because it 
appears to be a more useful type of information in helping increase user 
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confidence in the system. Indeed, as suggested by several participants In 
their written comments, there is a question concerning the extent to which 
presentation of justification should be separated from the system's 
conclusions. Some believed that justification should be included by default 
as part of the system's conclusions, while others preferred to have control 
over what information to receive. In any case, the results further suggest 
that users should be given the flexibility to decide whether justification 
will be automatically presented with the advice being offered by the 
system, much as they have a choice with conventional interfaces capable of 
providing both menu- and command-driven dialogue structures, or both 
tabular and graphic presentation formats. 

Developing an ES explanation facility with the capability to provide 
justification knowledge will be resource intensive. Many issues must be 
addressed, including the acquisition of valid domain knowledge, its 
internal representation, and automatic generation and presentation of 
comprehensible justification as explanations. Currently, a "canned text" 
approach remains the most efficient implementation technique (Wexelblat, 
1989). The approach, similar to what was used in this study, offers 
predetermined natural language responses by anticipating possible user 
queries. Because the system has no control over the content of the 
explanation, however, this approach is likely to cause serious updating and 
maintenance problems over time. In the long run, newer theories such as the 
one that treats explanation generation as "reconstructive problem-solving" 
promise to provide much more flexible and maintainable explanation 
facilities (see Wick and Thompson, 1992). 
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1 While the focus of this paper is the impact of explanations on ES users, 
an explanation facility has obvious value to ES developers. Validating and 
maintaining an evolving and expanding ES are non-trivial tasks, and an 
explanation facility can help developers recall the encoded problem-solving 
knowledge, identify faulty reasoning steps, and locate errors (Neches, et 
al., 1985). 

2 While the user's need for explanations stems from the unique 
characteristics of ES applications, the ability to provide explanations is 
admittedly technology driven. The typology adopted in this study Is 
strongly influenced by the rule-based tradition, one that remains the most 
popular and intuitive method of representing and reasoning about domain 
knowledge. Newer technologies, such as belief networks and neural networks, 
have been able to achieve expert-level performance without a cognitive 
model of the human expert. However, because the internal structures of 
these systems bear little resemblance to problem-solving expertise 
understood in human terms, it has been substantially more difficult to 
explain their reasoning processes to the user (Zehedi, 1993; but also see 
Suermondt, 1 992 and Klein, 1 994, for progress being made on explaining 
quantitative decision-making models). 

3 Definitions, which Include descriptions of terms and descriptions of 
differences between terms, are sometimes referred to as explanations (see 
Nettler, 1970 for a detailed discussion of explanation by definition). 
However, since a definition of something usually involves straightforward 
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information retrieval, not reasoning, the field has not considered it a 
major issue of explanation. 

4 Many ES development tools use a trace facility to answer two types of 
questions: WHY and HOW. The WHY question is typically interpreted as "why 
did you (the system) request that data," and is answered by showing the 
conclusion of a rule that can be inferred from the data requested (the goal 

to achieve). The HOW question, on the other hand, typical means "how did 
you reach that conclusion," and is answered by showing all the data used to 
lead to the conclusion. These interpretations, however, do not account for 
all possible user intentions and are therefore inadequate as a method of 
classifying explanations. See, for example, Lehnert (1978), who proposes 13 
types of questions for story understanding, including different kinds of 
why and how questions, and Graesser and Murachver (1 985), who identify 
seven question functions and three statement categories to form a taxonomy 
of 21 question types. 

5 As a result of this recurring relationship, Ehninger and Brockriede 
(1978) propose a simplified model known as a unit of proof, which includes 
every element of Toulmin's model except backing. 

6 Commercial expert systems for financial planning applications typically 
use such an input strategy because of large quantities of client data. 

7 The remaining three participants, all senior-level auditors, were more 
skeptical about the value of the system, mainly because they felt they 
could do better than what the system could. 

8 Note that a warrant is still a necessary condition, but not a sufficient 
condition (see Mackie, 1980). 
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